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ABSTRACT 


The U. S. Navy, in fulfilling its mission, requires a tremendous 
amount of skilled manpower. A significant percentage of this population 
1s frequently assigned outside their respective skill areas. The re- 
Sulting skill deterioration incurred during these assignments affects 
the amount of retraining required to reestablish currency and insure 
fleet readiness. This study suggests potential approaches for determining 
skill loss as the initial step for decision-makers in specifying the 
degree of retraining necessary. To this end, an extensive literature 
and agency survey of skill retention and related topics was conducted 
to identify areas wnere further research is required. Also, two con- 
ceptual models of possible retraining systems are formulated and dis- 
cussed. Finally, recommendations are submitted which have implications 
in the areds Of Reserve training, manpower planning, and personnel 


Management. 
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IRENE RODICTIOÓN 


B. BACKGROUND 

Between 1964 and 1969, annual federal expenditures on 
manpower training and development programs increased from 400 
MOn a pproswmately 2.2 billion dollars. In 1972, the 
United States Office of Education was budgeted in excess of 
e Dll dollars, and allocated roughity 800 million 
dollars on instructional materials and media alone. One 
Mano UU. S. comporation [Holt, 1963] reports spending 75 
million dollars on salaries of employees undergoing some form 
Fi eeraining Or retraining. The indicators pointing towards 
increased emphasis on training are limitless, and it is an 
appropriate follow-on to briefly investigate the causes for 
the increased activity in the training area during the last 
two decades. 

Perhaps the strongest motivation behind the intensification 
of training techniques has been the impact of increasingly 
complex technology on the total work force. The ensuing 
reduction in worker effectiveness resulting from a lack of 
knowledge of new technologies can be succinctly described in 
one word - obsolescence. What may be considered innovative 
today will most likely be considered antiquated in a few 
short years. Lukasiewics [1971], in his study of the engine- 
ering profession, has stated the useful productive life of 
an engineer after graduation is only five years unless 


adequate retraining is provided. Associated with the notion 
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of obsolescence is the impending shift of the need for un- 
“l labor te a highly skilled or multi-skilled labor 
Terce. 

In addition to the consequences of advanced technology, 
the rate of unemployment determined by economic fluctuations 
also necessitates the need for greater emphasis in training/ 
retraining programs. To this end, the Manpower Development 
and Training Act (MDTA) of 1962 was established and charged 
with the responsibility of initiating large scale training 
systems to help curb unemployment by offering retraining as 
an avenue of reentry into the work force. In the first ten 
years of operation beginning in 1963, MDTA programs have 
accounted for 4.7 billion dollars in federal funds [Mangum 
and Walsh, 1973], with a commensurate improvement in employ- 
ment of those individuals involved with the program. 

Inherent with industrial growth and technological ex- 
pansion is the need for a flexible work force. Generally 
Brreribedras Occupational mobility, this flexibility manifests 
itself in the shifting of jobs both within and between skills, 
and is a necessary evil in the realization that without proper 
Sci OE skills, the result will be an increase in un- 
employment. Taylor [1968] indicates that members of the 
American labor force change jobs on the average of every three 
te live years. This vast occupational mobility can prove to 
be an asset if properly channeled via appropriate retraining 
practices, or an unmanageable liability if left unattended. 

Retraining, as the reader will have noticed, has begun 


po requentliy reappear in the discussion thus far, and not 
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Umeuly sem Zn integral part of any effective training program, 
I r s jo p l C Ing OF skills is as important as the more 
conventional courses which enlighten the employee as to how 
to operate in his present situation, but fall short of sup- 
plying him the knowledge of how to cope with a changing en- 
vironment. Ida Hoos [1965] has illustrated this shortcoming 
Dy stating: 

A survey of industry has uncovered the fact that 

retraining on a larçe scale for workers threatened 

with skill obsolescence is rarely undertaken. 

Courses in safety practices, salesmanship, and 

management development prevail amonç industry 

sponsored endeavors. Indoctrination and orien- 

tation are commonplace; specialized instructions 

given by equipment vendors is a favoured practice. 

But programmes specifically designed to provide 

employees with new skills for job security in 

anticipation of technological change are exceedingly 

ie requcmt.. 

This is not to imply retraining is the panacea for worker 
obsolescence and unpredictable economic fluctuations, but 
facher to emphasize the relatively new field of skill mainte- 
Mance and rehabilitation in conjunction with the more orthodox 
training techniques now in use. Retraining takes place both 
aaa specific skill and/or entirely exclusive of a pre- 
e earned skill. The former entails such descriptors as 
upgrading of skills and keeping abreast of one's field, while 
the latter is typified by complete acquisition of a new skill 
and would hopefully include some positive transference from 
r Or O perience. 

betememmoving Om to a discussion of terms relevant to 


G c sit is more than fair to ask about the gains to be 


realized for the large expenditures certain to be required 
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for retraining programs. Perhaps the most numerative benefit 
is that such expenditures are both a social and economic 
investment toward adequate maintenance of the labor force. 

At the federal level, reductions in unemployment after re- 
training are readily apparent. Somers [1968] states that in 
almost all the surveys made on rehabilitating the unemployed, 
75% of the trainees were placed within their new skill 
specialty. In a study of post-retraining in West Virginia, 
it was found that the trainee recovered personal costs in 
four months; and the government was able to recoup its invest- 
ment within one year. At the corporate level, an effective 
retraining system facilitates job security and instills a 
sense of loyalty in the work force towards the organization. 
In addition, company employees with established service have 
proven their reliability, and it would be both wasteful and 
entail some degree of risk to hire outside personnel to fill 
new positions. Finally, the corporate image is enhanced if 
an Organization has a reputation for retraining internally to 
fill positions created by new technologies before resorting 
to external resources. Both quantitative and qualitative 
factors, as are mentioned here, influence the costs and 
returns to be expected and must be considered before a 


decision about the scope of retraining can be made. 


BD. CEE Toy AND DISCUSSION OF TERMS 
Although costs and benefits of training are the funda- 
mental parameters used in decision making, the initial under- 


standing for the need to train or retrain must take seed in 
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a discussion of basic terms germane to the general area of 
training. These terms will also serve as descriptors and 
Bere more speelrically the direction this study will 
Follow. 

Learning theory has evolved continuously along with man 
from the days of Aristotle and is an internal process in 
which the individual digests incoming stimuli and effects 
changes in behavior. The process that brings about learning 
is called training. Training processes must include the 
material to be learned, the method to best stimulate learning, 
and the evaluation of the resulting modified behavior. Since 
the areas of learning and training have received broad appli- 
cations, several related terms specific to this studv are 
presented and discussed. The following terms also provide a 
rough chronology of the processes involved herein. 

1% Skill 

Skill has been defined somewhat nebulously by a 
variety of sources which makes discussion of the concept less 


trivial than it would first appear. One source is the Oxford 


New English Dictionary whose definition reads, "practical 
Knowledge in combination with ability." Another is “ease, 
s 1 aAa precision (usually) of muscular action," found 


in Dictionary of Psychology [Drever, 1965]. A perhaps more 
useful definition was presented by Guthrie [1952], "Skill 
consists in the ability to bring about some end result with 
maximum certainty and minimum outlay of energy and time." In 
Eneseontext Of this study the term skill needs to reflect 


Pozscoriy a Manual dexterity, but also intellectual processes. 
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Salvendy and Seymour [1973] break these specialized abilities 
n n our divisions of labor: 

a. Sensory reception of stimuli. 

Be Mental perception and organization of infor- 

mation necessary to perform the task. 
en Cognitive decision-making to organize the 
proper response. 

d. The response involving motor actions. 
The ability to perform a given skill, therefore, constitutes 
the marrying of these stages into one continuous operation. 
In essence, several skills may be used to complete a single 
task, and several tasks may be performed to complete a job. 
Thus, a skill will be strongly related to performance as 
defined by how well an individual accomplishes his job. In 
addition, skill used in later discussions reflects the in- 
m oj ability to perform a specific task. 

2m Nonutilization 

Given that a skill has been learned and employed 
effectively to some productive end, nonutilization will be 
the term used to denote a period of disuse of that skill. 
A tears period, the skill or set of related skills are 
Dot Practiced. A nonutilization period will generally refer 
to an extended amount of time between one and four years. 
Causes of such periods are varied and may include promotion, 
reassicnment, extended periods of schooling, or a change of 
employer. Although absenteeism for whatever reason is also 


woa of skill disuse, its time span falls too low on a 
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hierarchial level to be used synonymously with nonutilization. 
Nonutilization results in obsolescence and a reduction in 
technical effectiveness. 
I. Skill Deterioration 

Given that a skill has been learned and then followed 
a period G@iemenutilization, the resulting affliction is 
called skill deterioration. This term simply refers to a 
Bee sorseroficieney in performing a skill. Skill deterior- 
ation has the inverse connotation of skill retention. Several 
variables of skill deterioration have been set forth by Naylor 
and Briggs [1961]. The first deals with task variables, 
which can be classified as discrete or continuous (sequential). 
Next, there are learning variables which play an important role 
in skill deterioration, and refer to the method employed for 
Original skill acquisition such as part versus whole task 
approaches. They also account for the amount of training 
provided. One of the most influential variables is the length 
K ie retention interval or the period of nonutilization. 
As discussed previously, this is the period of no practice. 
Later sections will discuss in detail the underlying variables 
associated with the nonutilization pericd relating to the 
Brpessersaetiyities undertaken during this period. Finally, 
there are recall variables such as environmental and mental 
conditions at the time the deteriorated skill is once again 
required. These variables all interplay with one another 


to produce a level of skill deterioration. 





4. Retraining 

As stated earlier, retraining in the civilian 
Employment sector reters mainly to teaching a new skill to 
those unemployed. The three R's for a healthy economy seem 
to be relocate, retrain, and reemploy. This connotation is 
contrary to the one desired for this study. In keeping with 
the chronology of terms presented above, retraining denotes 
a period of relearning following skill deterioration. Tra- 
ditionally in the military, one is trained in the skill areas 
required every time a new assignment or new equipment is 
mo ıjved. If, for any reason, a period of nonutilization of 
a particular skill is encountered, a period of retraining is 
required to reestablish the skill and alleviate the effects 
Sis kill deterioration. Thus, in this study, retraining will 
not refer to learning a new skill; but instead, will refer to 


relearning and updating a previous or highly related skill. 
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PERO PS TATEMENT OF THE PROBLEM 


A. THE SKILEED LABOR FORCE WITHIN THE NAVY 

As was mentioned before, there is an impending shift of 
the need for unskilled labor to a highly or multi-skilled 
labor force in the civilian sector. This trend is also prev- 
alent in the military, and is produced by the same technolog- 
feat mpderse witch Make production in industry so complex. 
Beecoramig to Haber [1974], in his study of occupational 
structure, there have been significant increases in craftsmen 
and related occupation areas both in the civilian and military 
sectors. The numbers of occupations requiring skilled per- 
sonnel within the military have been increasing at an alarm- 
morate and comprise roughly 30 percent for the Army and 
Marine Corps, 5l percent for the Air Force and 75 percent for 
the Navy. The wide scope of the Navy's mission and the fact 
that ships (combatants especially) are far more complex than 
aircraft, somewhat explains the Navy's greater need for a 
Bacher ratio Of technical skill in both the enlisted and 
Officer forces. It is quite likely that these percentages 
will increase in the future, and the implication for retraining 
considerations becomes obvious. Unless viable retraining 
programs are established, future fleet readiness may be jeop- 
ardized. 

The DOD Military Manpower Training Report for FY 1976 
indicates the Navy spending 878.5 million on specialized skill 


training of approximately 620,000 personnel with an average 
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school length of 22 days, which leads to a cost per student 
s 05 dollars. These figures exclude flight training, 
Best craining, Officer acquisition and professional 
development education. Data for rehabilitative training 
(retraining) is unavailable, but on the basis of the figures 
presented, we can assume retraining costs to be at least as 
high per student day as for initial training. It would then 
be in the Navy's best interest to construct cost effective 
retraining programs to maintain an adequate supply of technical 
personnel throughout the Navy's organizational structure. 
However, substantial groundwork is necessary before the re- 


training question in the Navy can be answered. 


B. CATEGORIES OF NONUTILIZATION 

The majority of Navy personnel are highly trained, and 
as high as 25 percent are assigned outside their skill areas 
ama require retraining upon return to their occupational 
Erreralrtes, The status of the individual during nonutili- 
zation will have a direct bearing on the amount of skill 
deterioration encountered. For example, personnel on inactive 
reserve status will incur less skill deterioration if his 
Secupation in the civilian sector is closely related to his 
s learned skill in the military. In order to reflect 
the entire Navy organizational structure, this study will con- 
Sider the following categories of nonutilization with respect 
to deterioration of a previously learned skill: 


I heme duty Outside of Skill area, 
But still within the Navy: 


20) Active duty with another Service. 
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3) Navy veteran with obligatory reserve 
service. 


4) Navy veteran without obligatory service. 

accion teo categorizing the types of nonutilization, 
it should be emphasized that the Navy reserve force (selected, 
ready, and standby active) of roughly 260,000 personnel repre- 
S or portion of the target population. Since 1971, 
the importance of the reserve structure has greatly increased. 
Meee ilitany Service Act of 1971 and the Defense Authorization 
Act of 1973 established the "Total Force Concept" which tasked 
the reserves with the responsibility of augmenting the active 


memes In yeu of conscription. 


C: Sli, DEWERTORATION 

pemee Valblous periods of nonutilization occur in approxi- 
mare ly 23 percent of the Navy's total enlisted force, the 
resulting deterioration of skills poses a significant problem 
to the Navy. Before skill deterioration can be combated, its 
variables must be identified and studied. It would seem 
reasonable to assume at this early juncture of the study that 
several factors will be found which impact differently upon 
urios areas. For example, skill type, length of non- 
Aa oniperiod, and type of duty during the nonutilization 
period are several of these relevant factors. The scope of 
the problem dictates, therefore, that an investigation of 
these variables is not only necessary to identify those which 
are applicable, but additionally is needed to determine the 
effects of these variables on skill degradation. Optimistically, 


Baeoescuay will point toward a future effort to develop a 
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system to quantify relevant factors which predict the amount 
of skill deterioration experienced over varying periods of 
nonutilization for the purpose of deciding how much retraining 


will be required. 


D RETRAINING REQUIREMENTS 

Once it is known how much skill deterioration can be 
expected to occur, the area of retraining can be investigated. 
When personnel return to previously held billets following a 
period of nonutilization for whatever reason, they must first 
ceive a period of retraining. Currently, the Navy lacks 
sufficient data to determine how much and what kind of re- 
Sealing 2S required. This situation leads to decisions 
Meesertoing elther too much or too little retraining. The 
former alternative results in decreased manhours for the 
fleet and increased training costs while the latter decision 
leads to decreased fleet readiness. The decision-maker's 
problem can thus be simplified to one of obtaining suitable 
information to enable him to make the trade-off between re- 


training costs and readiness. 


B. SUMMARY 

In summary, the total problem can be stated concisely as 
AOS Tae u. S. Navy, in fulfilling its mission, requires 
a vast amount of skilled manpower. A significant percentage 
of this population is sometimes assigned outside respective 
specialized areas. The resulting periods of nonutilization 
lead to skill deterioration. Upon reassignment, retraining 


is necessary to reestablish currency and ensure a high degree 
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of fleet readiness. Simply stated, the ultimate question 

is how much retraining is necessary. The scope of this study 
will seek to provide the initial steps toward providing the 
information necessary for decision-makers to determine the 
degree of retraining. This will be accomplished by a review 
of the literature and current projects being conducted by 
agencies that relate to the problem stated herein. The next 
step will be to identify the areas where further information 
is needed and to identify agencies where subsequent studies 
might be conducted. Finally, conceptual models of how the 
skill retention problem might be handled will be presented. 
Results from this effort may have implications in the areas 
of training and readiness of the Reserve, length of service 


requirements, and mobilization manpower and personnel planning. 
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PEN ORMATITON AD SOURCES 


A, SCOPE 

Skill retention can be affected by a myriad of variables 
and conditions, some of which are extremely hard to quantify, 
and attempting to be all-inclusive would produce a literature 
review of unwieldy proportions. Therefore, an effort has been 
made to review only those studies which are amenable to pro- 
Ang insight for the Navy's particular skill deterioration 
and retraining problem. In essence, emphasis was placed on 
memaresswalen dealt closely with operational conditions. Of 
higher concern was job-oriented procedural, perceptual, and 
cognitive skills rather than more simplistic motor skills. 
As the overview developed, identification was made of areas 
where sufficient amounts of information were lacking and 
warrant additional external effort. The skill deterioration 
and retraining approach further structured this review and 
will continue to be the common thread proliferating through 
As Paper. Le was the intent of this search to cover relevant 
studies for supplying the Navy with data of a recency of five 
to ten years. This survey involved formal searches with the 
following sources: 

INE a lpostoaraduate School Library, Monterey, CA. 

2) Defense Documentation Center (DDC) 

3) National Technical Information Service (NTIS) 

4) Psychological Abstracts Search and Retreival (PASAR) 


5) Human Resources Research Organization (HUM RRO) 
Lv brary, Carmel, CA. 
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Additionally, an agency survey was conducted and included 
several human resource laboratories and research activities 
in the Navy, Air Force, Army, and civilian sectors. In all, 
16 agencies were visited, while many others were contacted by 
telephone. A comprehensive discussion of the organizations, 
contacts, and related studies/projects is presented immedi- 
ately following the literature review. 

Finally, the areas identified as having a paucity of in- 
formation, or gaps in available data, will be discussed further 
wenes Of determining the criticality of such gaps and for 


suggesting areas of further study. 


B. ETERATURE REVIEW 
T. Skills 

Since one of the purposes of this paper is to ex- 
amine previous literature which may be of interest to skill 
retention in the Navy, the first area of investigation must 
be the skilled job areas themselves. It would seem reasonable 
CoN assúme at this point that any conceptual solution(s) to 
the problem must e anar a ls is or Job skills AS 
important to know how skills are acquired, how they can be 
classified and how they can be assessed. These three areas, 
A o acne organization Of this section of the literature 
review. 

a. BeEGuLGiclOn OL Skills 

This brief section serves as a background into 

some problems, variables, and experiments dealing with the 


learning of a skilled task. Bilodeau [1966] edited a collection 
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See padpers On Skill acquisition which provides an excellent 
starting point. The first chapter contains a paper [Irion, 
1966] which reviews the history of skill acquisition research. 
Jones [1966] then treats individual differences in learning 
and makes the point that genetic contributions are more prom- 
inent in the early stages of practice. Next, Fleishman [1966] 
sewes Glace basic abilities underlie skill levels. These 
basic abilities are cultivated early in life and remain constant 
thereafter because of overlearning. The abilities required 
For a particular skill change as the amount of practice in- 
creases. For example, verbal and spatial abilities decrease 
as motor abilities (kinesthetics) increase. Ina later article, 
Fleishman [1971] found that skill acquisition may be enhanced 
by concentrating on the abilities useful for final proficiency 
while deemphasizing those abilities used in early stages of 
learning. 

Craig and Bittel [1967] identified the require- 
ments of learning which include motivation, an appropriate 
stimuli, a required response, and confirmation in the form of 
reward or feedback. Heimstra and Ellingstad [1972] identified 
Ben mem izlables as being practice conditions; the nature 
of the material; prior experience; and again, motivation. 
individuawedieterences also are a factor affecting skill 
acquisition which has led to a vast amount of research with 
implleations for selection and training of skilled personnel. 
They also have found whole methods superior to part methods 


in skill learning. This means that a skill should be presented 
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and practiced in its entirety and not divided into segments. 
Finally, they stress that feedback or knowledge of results 
probably the single most important factor in skill acqui- 
See On. 

The importance of sensori-perceptual and 
decision-making components of occupational skilled performance 
Powemepm@asized by Salvendy and Seymour [1973]. Skilled per- 
formance involves sensory reception of information, perceptual 
ezganızatıon Of this information as it relates to the task, 
cognitive decision making to determine the necessary movements, 
and muscular action which accomplishes the task. Feedback is 
a vital input to the decision-making process if skilled per- 
formance is to be continued. They also reviewed some early 
work in the area of acquisition of occupational skills and 
reported the following findings: 

1) Learning speed improves as the size of the unit com- 
prising the task increase; 

2) Skilled performance relies on kinesthetic feedback as 
well as visual inputs; 

3) Comparison of learning curve plots is useful for 
evaluation of training methods; 

4) Skilled performance is a function of early practice 
patterns that can be associated with other similarly 
learned skills. 

Thus, it appears such variables as feedback, 
mig@vidual basic abilities, aptitude, and structure of the 
ie re situation itself all have considerable effect on 


skill acquisition. There have been many others who have 
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investigated this area, but it is hoped that this brief review 
furnishes a flavor of the types of ideas that have been studied. 
DE Gessır lestiomvor,skirls 

The mext logical step taken in a discussion 
of skills is the work done in the area of skill classification, 
K referred to by the term, taxonomy. The conceptual 
sections of this thesis will undoubtedly necessitate an effort 
eemelassifty various Navy skills, and it is therefore important 
to briefly review the efforts made toward this goal to date. 

Ze should not be surprising that a majority of the skill tax- 
onomy work has been motivated by the military. 

At this point it is both necessary and conven- 
ient to clarify several terms used to describe various hier- 
aea es Of classification. First, on the highest level, is 
vocational analysis which encompasses several occupations 
within an industry. A military example would be analysis 
pertaining to all squadron maintenance personnel. The next 
classification level is the most applicable to this study 
and involves occupational or job analysis. Since this thesis 
is to have implications for manpower and planning, information 
specific to individual billets is useful for making evaluations 
of skill deterioration and decisions about retraining require- 
ments. The term skills analysis is also used synonymously 
Batch occupation and job analysis. Finally, at the lowest 
level, is task analysis which is concerned with each segment 
of the job. The emphasis here is on the sensori-verceptual 


and psycho-motor processes involved to perform each job 
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túnetion. The remaining portion of this section is devoted 
Pema rier review of the literature on task classification and 
analysis. 

Helm cam nm S rlingscad [1972] refer togjob 

analysis as being useful to discover the behaviors necessary 
BOr skilled performance. Types of job analysis discussed 
include interview, questionnaire, work participation, and 
critical incident methods. Another pair of authors [Salvendy 
and Seymour, 1973] talk about job analysis in terms of answer- 
ing the question, "What does the worker do?" Analysis con- 
siderations must include psychological, physiological, physical, 
Ana dustrial engineering disciplines and approaches. With 
these disciplines in mind, several methods of analysis are 
presented under each category while all methods have in common 
the following items: complete job description, list of respon- 
sibilities, types of equipment used, and condition of worx. 
On the other hand, task analysis poses the question,"How does 
the person perform his job?"; which must be answered prior to 
the development of a training program. They also discuss the 
main classes of occupational skills which include handwork, 
handwork with tools, single-purpose machine work, multi-purpose 
machine work, group machine work, and nonrepetitive work. In 
order to be utilized in the training and retraining of workers, 
task analysis should be applied to performance which has 
reached what is referred to as the experienced workers' standard 
(EWS). 

eneie .ensensane Mills [1967] in their study of 


complex systems operators found that there was little data on 
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Operator skill requirements in a military operational environ- 
ment. The existing data, for the most part, came from simu- 
lated operations or simply from armchair conjecture. However, 
they maintain it is possible to obtain criteria for human 

er formance under actual operational conditions. Also, they 
call for increased standardization of skill classification 
systems. In this regard, one of the more widely used taxonomies 
is presented in Table I. The advantages of this taxonomy are 
its ease of use and comprehensive coverage of specific be- 
haviors. On the negative side, a disadvantage is that the 
best of specific behaviors is by no means exhaustive, making 
imam ICU IL and in some cases impossible, to fit all military 
personnel activities to this taxonomy. 

Also, Fleishman [1967], one of the foremost 
researchers in the area of skilled operator performance, 
completed a study on performance assessment related to a task 
taxonomy. Although there has been a large quantity of task 
analysis data collected in the past, new systems differ sig- 
nificantly with respect to application, mission, and technology 
making prior analysis inapplicable for the problems of skill 
[iene ttiecatien, training, and performance, now and in the 
coa discussed the need for a task taxonomy, and stressed 
PHEOuGiowe mrs work that the taxonomy must be founded on an 
empirical approach. To this end he identified the following 
eleven psychomotor factors: 

1) Control Precision - fine muscular adjustments usually 


in large muscle groups. 
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Processes 


Activities 


Speen ie 
Behaviors 





Pererceptual 
Processes 


2 Mediational 
Processes 


Communication 


4 Motor 
Processes 


Table 1. 


searching for and 
Receiving Information 


2 Identifying Objects, 
Actions, Events 


1 Information 
Processing 


2 Problem Solving and 
Decision Making 


Processes 


| Simple Discrete 
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Classification of Behavior 


Detects 
Inspects 
Observes 
Reads 
Receives 
Scans 
Surveys 


Discriminates 
Identifies 
Locates 


Categorizes 
Calculates 
Codes 
Computes 
Interpolates 
Itemizes 
Tabulates 
Translates 


Analyzes 
Calculates 
Chooses 
Compares 
Computes 
Estimates 
Plans 


Advises 
Answers 
Communicates 
Directs 
Indicates 
Informs 
Instructs 
Requests 
Transmits 


Activates 
Closes 
Connects 
Disconnects 
Joins 

Moves 
Presses 
Sets 





Processes 


Activities 


2 Complex/Continuous 


Source: Christensen and Mills 


Tables 


Classification of Behavior (continued) 


2 


SPECIFIC 
Behaviors 


Adjusts 
Aligns 
Regulates 
Synchronizes 
Tracks 





I 5 Nmb Coordination - found in tasks requirime 
Simultaneous control of hands and/or feet. 
3) Response Orientation - involves rapid visual discrim- 
ination and response in psychomotor tasks. 
4) Reaction Time - the speed of response to a stimulus. 
5) Speed of Arm Movement - involves simply the speed of 
gross arm movements and is independent of reactor time. 
6) Rate Control - Ability to trace a constantly changing 
target in both dimensions of speed and direction. 
7) Manual Dexterity - involves directing large objects 
with arm-hand movements under speed conditions. 
8) Finger Dexterity - involves directing small objects 
using, primary, finger movements. 
9) Arm-Hand Steadiness - ability to maintain a desired 
position with precise arm-hand movements. 
10) Wrist, Finger Speed - little used factor measured by 
“tapping” a pencil on paper. 
11) Aiming - involves eye-hand coordination in hitting a 
target with a finger. 
Additionally, Fleishman has identified the following nine 
n r of Physical proficiency: extent flexibility, dynamic 
flexibility, explosive strength, static strength, dynamic 
Strength, trunk strength, gross body coordinator, gross body 
equilibrium, and stamina. Both sets of factors account for 
the majority of common variance across a wide variety of 
tasks. However, at the same time, he stressed that these 
lists of factors were by no means exhaustive. Fleishman's 


approach for development of a behavior taxonomy was presented 
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in four phases; analysis of real tasks, synthesis and testing 
ot tasks, definition of behavior categories, and performance 
evaluation. Ina later paper [Fleishman, 1972] on human 

per tormance, he conclüded from studying individual differences 
in learning, that development of a skilled performance tax- 
onomy germane to applied problems should be centered around 
experimental and correlational methods. 

The Navy currently is using several classifi- 
cation systems for task analysis, and the primary system is 
the Navy Enlisted Occupational Classification System (NEOCS). 
The system was established to identify personnel and manpower 
requirements, facilitate effective manpower management, provide 
for personnel career welfare, and provide uniform terminology 
o enlisted occupational abilities, training, and experience. 
Recently a study of NEOCS was undertaken to identify problem 
areas and recommend improvements ["NEOCS Study," 1974]. 
Problems discovered were its unwieldy size due to over special- 
ization, lack of easy adaption to new enlisted skills, abundance 
u I S Int Skill definitions, amd failure to provide a 
meaningful base for advancement. 

Mes sence ene classifi ation Of skills Snould 
be job oriented and based upon carefully developed taxonomies. 
Piso. sinee the ultimate goal is to determine skill loss, the 
classification scheme should reflect the degree of importance 
f C of each task. Performance can then be measured 
as a function of critical task completion based on a hierarchial 


Beceem of classification. Finally, continuity should be 
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maintained by developing a standardized system amenable to 
both training and assessment communities. 
El Skill Assessment 

An investigation of the measurement of skill is 
m I r for the purpose of providing background information 
fomvett as identifying notable research in this area. It 
would seem relevant when discussing skill retention and re- 
training to include a survey of methods to assess skill pro- 
ficiency. To determine the extent of skill degradation and 
the amount of retraining necessary, one must first be capable 
of measuring the level of skill proficiency. There have been 
a number of notable projects undertaken through the years in 
search of a reliable and efficient method to assess perform- 
ance. In the interest of brevity, the work deemed most relevant 
to the present problem is reviewed here. 

To begin, Salvendy and Seymour [1973] define 
eae often used term, criterion. When discussing behavior or 
performance measurement, the term criterion is frequently used 
mpemcenote a standard of evaluation. A criterion referenced 
ct is one in which an individual's performance is evaluated 
Mdainse what he must do in order to successfully complete a 
MS IO the Contrary, a normative referenced test compares 
mend aual s performance to that of the group taking the 
same exam. A method of measurement qualifies as a criterion 
reference only if it is relevant to the skill being evaluated, 
Eeee en identify individual differences. Three types 
of criterion measures are ranking, counting, and establishing 


standards. 
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Christensen and Mills [1967] contend that 
because little operational data is utilized for performance 
evaluation, simulated operational experiments may never really 
measure the criteria associated with actual job performance. 

TE such data is to be utilized, it must be highly correlated 
with operational data. They stress that although performance 
deta collected under a Operational conditions is more 
difficult and may not be as "pure" as laboratory data, its 
utility far outweighs the disadvantages. It is recommended 
Bret systems be designed with the capability to efficiently 
collect periodic performance data. 

Fleishman [1967] has been one of the prominent 
pioneers of performance assessment. His work in this area 
was begun by trying to solve the military problem of predicting 
pilot performance. While based on empirically derived task 
taxonomies, tests should be evaluated on the amount of per- 
formance change they can detect. However, at the same time 
they must be capable of measuring across various experience 
levels. He recommends use of interim performance measures 
validated against operational performance, while task taxonomies 
are being developed. More recently Fleishman, et. al. [1973], 
developed guidelines for a research program on human perform- 
ance. While focusing on identification and evaluation of 
critical performance tasks, the authors discuss human reliability, 
performance optimization aids, performance standards, and skill 
and task structure. Several of these areas have implications 
for the previous section on task classification. Performance 


Beandards imvolve development of specific, dynamic criterion 
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for measurement of performance, Thus, a quantitative criterion 
based assessment tool should be utilized, vice a subjective 
supervisory rating. The report sees a need for the Navy to 
incorporate two approaches into its performance assessment 
program: one approach is Fleishman's [1972] use of empirical 
correlations of task interrelationships, and the other is 
Alluisi's [1970] synthesized human function tasks. Because 
of its use of time sharing in a multi-channel setting, the 
latter approach 1S more operationally oriented. Once more, 
it is emphasized that assessment of performance must be as 
much like the actual operational job as possible. Points of 
common deficiency in this area are failure to consider complex, 
multi-dimensional, and sustained performance conditions; as 
well as motivation, general behavior, stress, and time-sharing 
tasks. Any assessment mode outside the operational environ- 
ment must prove to be a valid measurement under actual con- 
eat Ons . 

The U. S. Army Research Institute for the 
Pehayioralsend Social Sciences [Mailer, 1976], sponsored a 
study to determine the feasibility of hands-on performance 
tests for technical jobs. The study indicated when constructing 
any type Of performance measure, an objective must first be 
developed which consists of task description, standards Or 
performance required, and conditions under which the task must 
be performed. A literature review was conducted which showed 
trends in performance measurement shifting from normative 
reference tests to criterion referenced tests. Examples of 


criterion references are the time to complete a task, the 
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quality of the output, the correct use of equipment, and the 
use of proper procedures. They constructed and successfully 
used a hands-on performance test for an electronic maintenance 
MOS. Possible problem areas included lack of equipment avail- 
ability for testing and time constraints on the evaluation 
Period. The latter may detract from total job performance. 
Mae results of the effort have been published in a guidebook 
PerwcOnstructing and administering criterion referenced tests 
(Sweezy and Pearlstein, 1975]. Several other military agencies 
are currently engaged in projects on performance measures and 
these will be discussed in a later section of this study. 
Summarily, the assessment of skill should be 
operationally measured, utilizing the same criteria emphasized 
during the training period. This criteria should reflect task 
si; Tiy and should be derived from operational data and 
not a controlled laboratory environment. If proper attention 
is given the method of assessment, the results will provide 
information for eventually determining retraining requirements. 
Dr Skill Deterioration 

Pes ergation/ into the deterioration- of technical 
skills comprises a major portion of the scope of this study. 
Bares section will review pertinent literature on the subject 
Se a retention and will also strive to pull together the 
Morane findings in the area to date. The subject of skili 
retention has been a relevent topic for study by both industry 
ene migitary. This popularity has produced several liter- 
ature reviews over the past fifteen years, the most recent of 


which is Prophet [1976]. Thus, to prevent redundancy of effort, 


of 





the past literature reviews will be discussed etnhereugn!v along 
with several significant references which will serve as a 
Background to the topic. 

bei”, Skill déterioraāation is most difficult to 
measure. Only by subtracting what is retained from what was 
originally learned, can one estimate how much information was 
lost. This is what Ebbinghaus did as early as 1885. His 
Ame tes resulted in the formulation of a “forgetting curve" 
which showed huge losses in retention shortly after the learn- 
ing period. As time increased the forgetting rate decreased. 

Since Ebbinghaus, others have found many more vari- 
ables affecting retention in addition to time. Four factors 
influencing retention are identified by Heimstra and Ellingstad 
[1972]. The first is the meaningfulness of the material. It 
is generally agreed that the more meaningful or the more 
organized the information is to the learner, the better it 
will be remembered. Second, the type of practice utilized 
in the training environment can have a positive effect of 
retention. For example, retention is enhanced by many small 
Practice sessions vice a larger massed session. The third 
deals with proficiency, where the level of mastery of a skill 
is strongly believed to be inversely proportional to the amount 
of skill deterioration experienced. Finally, a fourth variable 
of retention is its method of measurement. The following three 
such methods were discussed: 

Deran this situation the learner must reproduce 
what was learned previously. This is the least accurate 


method due to the amount of other factors besides 
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retention that could prevent the subject from expres- 
sing the correct response. 

2) Recognition - Here one must identify the correct 
response(s) from a field of incorrect distractors. 
The accuracy is dubious because the correct answer 
can be arrived at by guessing. 

3) Savings - This method requires the subject to relearn 
a previously known task while quantifying the number 
of repetitions required and comparing that to previous 
training sessions. This method is the most accurate 
measure of retention. 

Fleishman and Parker [1961] studied retention of 
motor skills. They varied retention intervals, type of initial 
FEeauming, and level or initial training. The most significant 
variable found was individual differences in the level of 


m s] training. The length of the interval for these types 


— 


= 


= 


o Skills was not an important factor 
There 1s One additional study [Armstrong, 1975] that 
bears mentioning when discussing the various variables of skill 
deterioration. The factors of retention are categorized into 
the following four areas: 
1) Type of tasks - Motor type skills are usually retained 
longer than verbal/procedural type skills. Examples 
of motor skills are cycling, swimming, and basic flying. 
Examples of a procedural task are electronic repair 
and computer programming. 
2) Types and amount of training - Retention seems to be 


proportional to the amount of original proficiency 
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and to the amount of organization in the Fr nung 
environment. Also, the likeness of the recall sur- 
roundings to the initial training environment affects 
ehe amount of retention. 

3) Retention interval - Retention generally decreases 
Over time. Skill decay, in addition, depends on 
what occurs in the nonutilization period. For example, 
Plemrypes "OL practice, if any, performed on the task 
or the learning of another moderately related skill 
Canwstontiveantliy affect retention. 

4) Retraining situation - Retraining time is directly 
dep ment ensche®proficieney of original training and 
on the length of the period of disuse. 

Since industry does not suffer from skill deterior- 
ation due to nonutilization of the same magnitude as the 
mary, their skill degradation efforts are focused around 
the problem of obsolescence. A modern Navy also suffers from 
this form of technological turbulence as well as nonutilization. 
Thus, this seems an appropriate place to briefly discuss this 
topic. Obsolescence is a form of skill deterioration although 
it results from an external source. It is not one's retention 
of a skill that changes in this case; rather, it is skill 
L jr eman S that vary. Dubin [1974] reported that very little 
research has been conducted in identifying the variables of 
obsolescence. Obsolescence is defined by many to be a re- 

n r in performance with time, which results from the 
development of new technologies since the worker's last period 


of training. Mali [1969] developed an obsolescence index: 
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OL = Current knowledge understood by personnel 
Current knowledge in the field 


In relation to training, it is suggested that obsolescence 

be defined as the difference between the knowledge and skills 
Saea current graduate, and the practicing individual in the 
fleet. 

As mentioned previously, there have been several 
erature surveys in the skills retention (or deterioration) 
area since 1960. Four of these reviews will now be discussed 
Merdeer tO lend brevity to this portion of the present skill 
retention overview. 

Perhaps the first milestone study was performed by 
Neto rana Briggs [1961] for the Air Force in the specific 
area of flight skills retention. Over 120 items were covered, 
but were predominantly research and not operationally oriented. 
In any case, the authors classified skill retention variables 
into task, learning, retention interval, and recall categories. 
Of primary interest here are the retention interval consider- 
ations. Although there is a paucity of data highlighting the 
effects of varying amounts of practice during the period of 
G s E On it was found that longer periods produced larger 
performance decrements and also, this relationship was found 
to be nonlinear, wherein the greater performance decrements 
Secur leacliy on, then taper off. Other significant findings 
were: 

Deore nous (tracking) tasks are retained better than 


discrete (procedural) tasks 
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2) The higher the degree of mastery dur mes Tat 
acquisition, the longer the retention of a skill. 
3) The greater degree to which a task is integrated and 
specified, the better it is retained. 
4) Whole learning, as opposed to modular, can lead to 
better retention, especially for more complex tasks. 
5) Measurement plays a key role in ascertaining the 
degree of retention. It is of great importance to 
insure the criterion used and measurement be critical 
moe ene successtul completion of the task. 
The Naylor Briggs review is quite comprehensive, but to a 
large degree 1S oriented toward verbal learning and memory. 
Of more concern in the present study are the procedural and 
psychomotor skills which are involved in Navy technical ratings. 
AO ROSS Mana TO "BB. Turner [1967] investigated 
the high incidence of skill loss when Navy personnel were 
assigned to non-rating related billets ashore. The initial 
Mase or the investigation was primarily a bibliographic 
survey, and unfortunately funding for a subsequent longitudinal 
study wasn't available. However, the initial report represents 
one of the first concerted efforts to provide the Navy with data 
eneretention of technical skills. In the statement of the 
problem, skill loss was defined as loss of a previously learned 
technical skill, and the important difference between skill 
loss and obsolescence was pointed out: 
The inability of personnel to perform assigned 


duties due to changes in equipment between previous 
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and present billet assignments [obsolescence] 
Should not be confused with inability to perform 
Aro Eo SRT loss througn lack of Opportune =o 
practice Job skills. 
Rose and Turner summarized the following points from their 
combined literature and agency survey: 

1) Discrete tasks involving sequential steps incurred 
considerable skill loss over time, while continuous 
tasks suffered no decrease in retention. 

2) Job skill research has been predominantly oriented 
E@wards Operational skills and not maintenance skills. 

3) Few reports exist on skill loss due to nonpractice 
of those skills. 

The next important skill retention review was con- 
Emeredsby G. R. Gardlin and T. E. Sitterley [1972] of the 
Boeing Company, working under contract for NASA in 1972. They 
separated retention variables into four categories: 
Mamo of training; (2) duration of retention interval; 
(3) task organization; and (4) task environment. In general 
Esel) rerenelon appears to be functionally proportional to 
amount of training, extent of task organization, and the de- 
gree of similarity between task environment and operational 
setting. Conversely skill retention, except in regard to 
continuous skills, is most likely inversely proportional to 
dra tion of the nonutilization period. A commonality shared 
by all four above mentioned categories is the level of skill 
Meeeniecmgust prior to the retention interval; °...the key 


Bale tor in predicting skill retention for a given no practice 
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interval appears to be the final level of skill acquisition. 
Other variables were seen only to modify this level." 

The authors also express caution about extrapolating 
retention data gained from controlled laboratory environments 
Mene Operational setting. Most of the literature involves 
the investigation of performance wherein the operator is con- 
cerned only with a single task. More often, an operator handles 
multiple tasks, and interference effects inherently increase 
the task difficulty. For example, an intercept operator 
aboard a modern fighter aircraft will perform three tasks, 
perhaps simultaneously: (1) track a target on a radar scope 
(continuous task); (2) vector the pilot towards the target 
(discrete task); and (3) arm and fire a missile (discrete 
task). The secondary task of vectoring the pilot can produce 
decrements in initial retention test performance in the primary 
Biase, cracking the target. It is in situations as just 
described where effective task organization and the use of 
high fidelity simulators can significantly improve retention 
and/or maintenance of skills. 

Gardlin and Sitterley go on to say that training/ 
retraining programs should be based heavily on critical per- 
formance dimensions. That is, essential task elements upon 
which the success of the mission depend must be identified 
and specifically stressed. In addition, any performance 
measure used should not be global in nature, but designed to 
determine whether or not the individual can perform those 


asks deemed critical to job success. This applies to the 
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broad spectrum of task skills, from those as complex as 
spaceflight to more mundane levels such as radar scope moni- 
mor ing. 

Prophes 11976], the last review to be cited, is 
definitely the most comprehensive literature survey on skill 
retention to be undertaken to date. In his study of long- 
term retention of flying skills in 1976, Prophet reviewed 
20 ttems published primarily in the sixties and seventies. 
He also discussed several previous literature surveys, in- 
Cluding Naylor and Briggs, and Gardlin and Sitterley. In 
addition to a most detailed literature review, Prophet pub- 
lished an annotated bibliography under separate cover, as a 
companion report to the survey. Over 120 items are synopsized 
and are grouped on a scale of relevancy to flight skill re- 
Pent lOn. 

As a first step in assessing the research activity, 
Prophet compares his survey with that of Naylor and Briggs, 
and Gardlin and Sitterley. Degree of recency (Table II) and 
amount of overlap (Table III) form the basis of comparison. 
Peomeenese extracted tables, it can be seen that Naylor and 
Memagsomeovered resources up to 1960, Gardlin and Sitterley 
focused on the sixties, and Prophet centered predominantly 
in the late sixties and the first half of the present decade. 
79 of 120 items in Prophet's review were not covered in either 
of the other two surveys, largely because of recency. This 
Mieten imereasing amount of activity in the skill re- 


tention field, but according to Prophet, not near enough: 
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Naylor & 


meee Of Publication Briggs 

1S) ae 0 
1970-1974 0 
65-7969 0 
1960-1964 1 
5217959 25 
50-1954 28 
1945-1949 5 
1940-1944 Jolt 
295-1939 14 
1930-1934 19 
2025-1929 6 
1920-1924 6 
0175-1919 3 
1910-1914 2 
1905-1909 2 
1900-1904 0 
2795217899 0 
l 0015942 0 
1 1939 1 

Total 123 

Table II. 


Gardlin & 


Sitterley 


0 
3 
62 


° / 


Le 


Prophet 
6 


40 
40 


25 


0 


— — — 


120 


Publications Reviewed - By Year of Publication 
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Author(s) Number of Items Reviewed 


Naylor only ELS 
Gardlin only 72 
Prophet only 79 
Naylor-Gardlin 5 
Naylor-Prophet 2 
Gardlin-Prophet 36 
Per Nor - Gardlın-Prophet ae 

Total >10 
Table III. Commonality of Three Literature Reviews 
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It would he a mistake to assume, however, that 

the problem [skill retention] is receiving 

adequate research attention, for of the several 

hundred studies included in this review, only 

a relative handful deal with skills of the 

magnitude of complexity of aircraft piloting, 

and very few deal with retention time intervals 

similar to those of concern to the USAF 

(1-3 years). 

PrPesnee Gdiscusses his findings in terms of the Lol- 
lowing three categories: (l) general retention factors; (2) 
= Or Skill factors; and (3) retraining factors. As in the 
Sitterley review, Prophet indicates the best predictor in 
determining performance after intervals of nonuse is the high- 
ceee vel Of Skill acquired prior to the interval. In regard 
Comte duration Of the retention interval, no adequate quanti- 
mer ve depiction Can be had from research to date. All that 
can be conclusively stated is the longer the period of no 
practice, the larger the decrement in performance. A hy- 
Beemesis by R. H. Wright [1973] that flight skill degradation 
Boma toicmine tae first 6-12 months of nonutilization 1S sup- 
ported somewhat quantitatively, but performance ratings were 
self-judgements by the operator population, not actual inflight 
Measurements. In trying to derive forgetting curves for 
combat pilots, M. B. Armstrong [1975] used returned POW's 
Pecausce of thelr Variant periods of inactivity. However, 
beeause Gf the nonhomogeneity of the group, and relatively 


few data points, inferences about the shape of such a retention 


curve would be somewhat risky. Of much interest 1s Prophet's 
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emphasis of interference factors during the period oO no 
practice. Intuitively, one could think any related skill 
practice would enhance retention, but this is not always the 
case. Prophet cites several proficiency flying studies in 
which pilots who flew light aircraft during nonflying assign- 
ments, and then retrained in a more complex aircraft, required 
Meise retraining than did pilots who had not flown at all. 
Clearly, there exist such interference factors in other pro- 
eedural related skills, especially within Navy technical 
ratings. 

In discussion of discrete (procedural) versus con- 
tinuous tasks, Prophet agrees with the consensus of reports 
indicating continuous tasks are retained better than discrete 
tasks. He goes on further to say that because of procedural 
task loading, instrument flying is more difficult to retain 
than visual flying. Finally Prophet highlights some pertinent 
retraining factors which are discussed in the retraining 
portion of the present literature review. 

By far the most exploited of the research areas 
discussed in the present literature review, skill deterioration 
G i result of a myriad number of factors, perhaps the most 
prominent being the amount of original proficiency acquired 
just prior to nonutilization. This idea seems to be a common 
Ano es reement.. Other considerations such as length ot 
nonuse, type of skill, interference variables and individual 
Eerrerences; impinge directly on skill retention; but not to 


the same degree as original proficiency. It would then appear 
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ne best antidote for skill loss is a high level of ae 
prior to an assignment outside a respective skill area. 
Or Retraining 

During the last two decades, there has been a great 
deal of effort at the federal level to cope with unemployment 
by establishing government/state supported training programs 
such as those sponsored by the Manpower Development and 
Training Act (MDTA) of 1962, and the Area Redevelopment Act 
(ARA) of 1961. In essence, endeavors in the civilian sector 
have dealt with training new skills to the unemployed and 
Moe metraining Or updating previously learned skills. In 
contrast, this study seeks to explore retraining on a basis 
which will have implications towards such things as retraining 
the selected, ready, and standby active reserves to augment 
the active duty force when necessary. Another application 
would be ascertaining the amount of retraining required of 
Sl individual returning to his unit after a period of temporary 
sonal duty (TAD) such as Shore Patrol or Master at Arms. 
The immensity of the manpower involved is indicated by a 
figure of just under 260,000 Navy reservists and a significant 
percentage of the active force on duty outside their skill 
area. 

Programs to retrain or update the skills of this 
large population upon reentry into active duty (i.e., during 
pert temmeangmentation Of «active forces) ame virtually non 
existent. However, some training techniques now being studied 
could be applicable. One such technique is that of generalized 


training, where some common denominator of mission, equipment, 
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or operational technique exists between systems. The U. S. 
Navy is now considering this approach in Acoustic Sensor 
Operation (ASO) training, where a multiplicity of systems 

and training hardware are now in use. Ina study sponsored 

by cee Naval Training Equipment Center (NTEC), R. W. Daniels 
and D. G. Alden [1975] propose to take advantege of the com- 
monality inherent across the broad spectrum of Navy acoustic 
sensor systems. The degree of commonality would depend on 
Operator task, skill, and knowledge required. This generalized 
Paamming concept could be used quite effectively in retraining 
areas, especially within a general mobilization plan for the 
reserves. As was mentioned before the effects of the "Total 


Force" concept and an all-volunteer Armed Services have 
placed the responsibility for augmentation of the regular 
service upon the reserve force structure in lieu of conscrip- 
tion. The Army was perhaps the first service to be affected 
and has taken steps to geographically realign its reserve 
Senpements to facilitate training and mobilization. A Table 
of Organization and Equipment (TO&E's) has been initiated to 
K (very Unit and individual to a specific mobilization 
force. This geographical realignment has done much to en- 
MemeGen ehestraining effort and could certainly provide a 
foundation for implementing generalized training techniques 
already discussed. 

Since there exists well established lines of autnor- 
ity and communication within the military, another option for 
retraining is available and already well developed. Systematic 


training, very much similar to programmed instruction, entalls 
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a careful examination of material, detailed breakdowns, step- 
by-step presentation and frequency feedback of results. In 
his study of technological change, Seymour [1966] emphasizes 
that careful classification (taxonomy) of the tasks involved 
is an absolute necessity for any training program. Taxonomies 
have been mentioned before and the Navy currently has the 
Pes-eurme! Qualification Standard (PQS) which closely parallels 
the intent of job taxonomy and provides step-by-step pro- 
Secures Lor most tasks in the Navy. PQS could be applied to 
m Cm components during the mobilization phase prior to 
active duty augmentation. 

In many skill areas, especially in the aerospace 
Field, both maintenance of proficiency and retraining require 
operation of extremely sophisticated and costly equipment. 
Several studies have investigated the feasibility of using 
less complex equipment for major portions of continuous and/ 
See retraining programs. The Air Force, in an effort to curb 
the high cost of large transport (C-141, C-5) proficiency 
Flying; used the T-37 trainer as a Low Cost Aircraft (LCA) to 
maintain flying skills. Unfortunately, some degree of task 
interference was experienced and coupled with a less than 
enthusiastic subject response, the LCA program was not suc- 
cessful. However, the concept is valid, but especially 
SiPtticult to apply to aircrew training. In a similar approach 
rh U. S Army missile technicians, D. L. Grimsley [1969] 
experienced a great deal more success than was had with 
transport aircrews. In a series of studies, Grimsley sougnt 


to provide performance rehabilitation using low fidelity 
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simulators in place of high fiedlity (actual uC at iO) 
devices which were more costly to operate. He found that 
the use of such low fidelity systems had no adverse effect 
on level of proficiency, retention, time to retrain, or 
wnether individual or group learning was used. Also, bear 
in mind the tasks required of missile technicians are extremely 
procedural in nature and would be representative of the tech- 
nical skills required by scores of ratings in the Navy. It 
Should also be noted that Grimsley considers a careful review 
of tasks to be taught be made prior to selection of such low 
Fidelity training devices. 

Perhaps the area which has received the greatest 
Apeenezon is the retention of flight skills, and not unduly 
so. In FY 1975, the various military services flew approxi- 
ftieotyeo.2 Million hours at a cost Of about 2.7 billion dollars. 
Most of this flying was for training to develop and maintain 
pilot proficiency as an integral part of military readiness. 
However, the figuring of this cost is easy, whereas the re- 
Sultant impact from an increase or decrease of flying training 
and readiness is not. To this end, literally hundreds of 
Studies have explored the relationship of skill retention and 
the myriad number of maintenance schemes intended to be cost 
SeeeCetve and Guarantee proficiency. Of the four major reviews 
mentioned already, Naylor and Briggs [1962], Rose and Turner 
MSI), Gardtin and Sitterley [1972], and Prophet [1976], 
Prophet's survey seems to be the most comprehensive and ob- 
viously the most recent. In his review of almost 120 studies 


and articles, and an accompanying annotated bibliography, 


53 





Prophet concludes the following about flight skills mainte- 
nance/retraining: 

1) Virtually all of the training required to reinstate 
basic flight control and procedural skills can be accomplished 
in modern flight simulators. In general, training devices 
can be used with high cost effectiveness in flight skills 
maintenance and retraining programs. 

2) The length of retraining programs is a function of: 

a) Careful specification of program objectives. 

b) Deliberate design of program to achieve the 
objectives. 

c) Adequate methods of measurement and quality control. 

d) Use of individualized, self-paced training techniques. 

e) Cost effective use of training devices. 

3) Individual operator experience. As experience in- 
creases, the pilot “learns to learn" and enhances performance 
over time. 

4) Concomitant with 3), there is a general decline in 
performance, starting in the late thirties or early forties 
of an operator's life. 

5) Basic flight skills (psychomotor) are retained well 
over time and are more easily reinstated that instrument 
arocedural) skills. 

6) Finally, Prophet discusses the concept of the "unviable 
pilot." Such a person may be unsuitable for retraining due to 
eencivemecrtods of nonflying, poor initial training, of 
degradation of some physiological or psychological variable 


deemed necessary to function as a pilot. Retraining programs 
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should seek to identify such individuals upon reentry to the 
skill area and channel them elsewhere. To try and retrain all 
personnel would be as ineffective as retraining none. Some 
quantitative measure should be used to screen out the unviable 
portion of the population before retraining begins. 

Even though Prophet's survey was concerned with 
Margie Skill retention, one must realize that flying skill 
depends upon both psychomotor and procedural subskills. Thus 
Prophet's conclusions will also have substantial impact in 
technical skill areas which are greatly procedural in nature. 

maaan Ssitterley 1197217 in a literature 
Seavey In Support of their investigation of astronaut skill 
degradation for NASA, point out the need for skill maintenance 
Ber long alrations of task inactivity. The extent of sched- 
uled on-board refresher training would depend on several 
variables such as the nature of the astronaut tasks, the time 
Since last performance, the degree (if any) of task over- 
learning, and the software/hardware complexity. As in the 
Prophet survey, Gardlin and Sitterley cite the lack of an 
adequate performance measure to quantify actual skill deteri- 
Sraeton. They go on to say any such performance measures 
skeuld be task oriented and conducted in an operational 
Send rather than an artificial laboratory environment. 

o oe and Briggs [1961], the earliest of Ene our 
major skill retention surveys discussed herein, dealt with 
verbal learning, as opposed to skills learning, and as opposed 
to skill retention field, and for this reason it was discussed 


ara (skill Deterioration) of this section. In regard 
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to retraining programs, Naylor and Briggs brought out, as 
long ago as 1961, the inadequacy of measurement methodology. 
Specifically, their concern centered around measurements over 
periods of retention, such as pre/post testing. There was 
then, and apparently presently, no consistent measurement 

of Job performance. Again this opinion is shared by all four 
major skill retention surveys. 

It is necessary that some essential ingredients of 
retraining be reemphasized at this time. It is well known 
mee seme Of the active duty population and virtually the 
s ir reserve Structure, train, retrain, and requalify on 
equipment in varying stages of antiquation. The Army has 
Some EO "grips with this problem and is currently outfitting 
their reserve components with up-to-date material Since 1969, 
Army Reserve Components have received over six billion dollars 
in new equipment. G. A. Leon [1975], in his discussion on 
the total force concept, implies the United States should 
equip its reserve forces with modern weapons and equipment to 
permit effective training and strength development. This 
should take precedence over supplying our allies with present 
generation weapons systems. Even the best of retraining pro- 
grams will be of little value without adequate operational 
and training material. Until such a time when adequate fund- 
ing becomes available for training equipment, exploitation 
cie Cee retraining factors should be explored. Use of 
low cost/low fidelity training devices, generalized retrain- 


ing concepts, and self-paced instruction are just a few 
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considerations. Since retraining appears to be proportlonal 
to the length of the nonutilization period, limits should be 
placed on the numbers of personnel assigned tours of duty 
cr IG their skill areas. 
4. Summary 

Before proceeding to the agency survey, a brief 
recounting of the more salient results from the literature 
Sin orcder. First, interest must be directed towards skill 
acquisition and what factors affect the learning process 
memas teedback, individual aptitude, and the structure of 
the training environment. Second, the variables of retention 
need to be considered as they provide feedback to the train- 
ing environment, and aid the decision-maker in determining 
retraining requirements. Amount of original proficiency, 
length of the retention interval, type of skill (e.g., con- 
tinuous or discrete), interference factors, and individual 
differences are but a few considerations relative to the 
retention area. Next, the method of quantitatively measuring 
skill loss via performance assessment is discussed. Proper 
assessment criteria must be selected and matched with train- 
Mmieertreria. Furthermore, the criteria should reflect task 
criticality and be based on operational vice laboratory data. 
Tt should also be generally noted that retention curves prove 
E a estimators, primarily because they are based on 
ES A te date points. Finally, the general retraining 
process is treated as it relates to rehabilitating a previously 
acquired skill. The use of both self-paced instruction and 


low cost, low fidelity training devices has proven to be 
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beneficial to reducing retraining time. It is generally 
agreed that the amount of retraining required is approximately 
proportional to the length of the nonutilization period, but 
Hemedenitatication of this relationship is yet offered. A 
relatively new idea, the generalized training concept, which 
takes advantage of system commonalities (e.g., the operation 
of various sensor systems) to retrain basic procedures, has 
met with some success and should help in reducing the high 
cost of retraining. As a last consideration, the idea of 
retraining all personnel upon reentry to their respective 
Skill areas should be scrutinized. Those individuals classi- 
fied as "unviable" should be screened out and not retrained. 
Previous inadequate performance or unusually long periods of 


Fonmucalization could mean inordinately high retraining costs. 


Be AGENCY REVIEW 

Beyond a review of the literature, the scope of this 
study includes a review of the agencies who are currently 
engaged in related projects or have experience in the areas 
ARAN deterioration and retraining. Several additional 
agencies were contacted that could not provide any useful 
information, and they will not be mentioned here. A vast 
majority of profitable contacts were made with military sources. 

The structure of the agency review will be to first iden- 
tify and give the location of the agency. Then the names, 
titles, and telephone numbers of points of contact within 
each agency are given. Finally, a brief description of the 
work and/or ideas of each agency relative to this study is 


discussed. 
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Chief of Naval Education and Training (CNET) 
NAS Pensacola, Florida 


Dr. Scanland, ACOS for Research and Program Development 
(AUTOVON 922-3466) 


Mr. Douglas Davis, Occupational Standards Training Programs 
(AUTOVON 922-3466) 


Mr. Hooprich, NAVPERSRANDCEN Liaison (AUTOVON 922-2621) 


Although CNET has no current program efforts devoted to 
research pertaining specifically to this study, there were 
helpful contacts made which merit attention for possible future 
reference. First, Dr. Scanland stressed the need for more in 
the area of assessing job performance which is a little under- 
s C rt From a training point of view, learning criteria 
reference measures should be patterned after Suitable job 
performance measures (JPM). JPM's are also good for helping 
to set standards and for instructional feedback. To accomplish 
this requires a synergistic training/performance system un- 
like the current diffuse subsystems that have no lines of 
communication between them. 

Mr. Davis is the JPM expert at CNET. He states that 
training curricula are developed from computerized job data 
gathered on Job Data Worksheets (Figure 1). These worksheets 
Bre essentially a method of task analysis. There are 640 
task action codes that identify specific Navywide skills. 

Mr. Hooprich is a liaison between CNET and several other 
Navy agencies. He is,thus, able to keep in close contact 
With all CNET projects and most training related projects 
throughout the Navy. He helped tremendously with this thesis 


effort and would be a good point of contact for any future 


efforts. 
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CNET Technical Analysis and Evaluation Group (TAEG) 


Nie Orrandor of lOorida 


Dr. Smode, Director (AUTOVON 791-5198) 

TAEG is not currently engaged in any projects directly 
related to this study. However, Dr. Smode had several ideas 
North noting on the subject of retraining. A possible approach 
to the problem should include the following three steps: 

1) Define the job requirements. 

2) Assess how well the worker can perform at the present 
time regardless of his nonutilization period variables. 

3) Retrain the individual to the required level. Histori- 
m collecting assessment, training, or skill deterioration 
data in an experimental setting has translated poorly to the 
operational environment. TAEG has had much success collecting 
actual operational data and believes this to be a much more 


Viable route of progression. 
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Director of Naval Education and Training (DNET) 
Arlington, Virginia 


Captain M. K. Mailhorne, Deputy Director, Programs 
Division (AUTOVON 222-4835) 


Captain Mailhorne is sponsoring the NPRDC study on the 
Performance Proficiency Assessment System, mentioned earlier. 
G I Or to his position at DNET, he is also the Director 
for Plans and Policy for the Naval Reserve Readiness Command 
headquartered in Baltimore, Maryland. Though the NPRDC study 
on proficiency is longitudinal in nature, Captain Mailhorne 
believes a shorter term answer to skill deterioration is 
necessary. Some "gross cut" measure which will provide 
decision-makers a basis for determining retraining require- 
ments for large numbers of skilled personnel, especially within 
the reserve structure. All too often, the lengthy longitudinal 
studies lose momentum when key individuals move on to other 
assignments. 

IA Ir nc ion with his position in the reserve community, 
Capt. Mailhorne has submitted a point paper on training require- 
ments to maintain proficiency in the Reserves. The adequacy 
Semeheworesent drill structure and active duty period is in 
question and a study is called for to: 

A determine the effects of various drilling and 

ACDUTRA combinations upon the proficiency of post active 

duty drillers in various skill categories/field/applica- 

tions... The intent of the effort would be to obtain | 

'ball park' information on the basis of which to exercise 

profesional judgement, rather than to create precise 

measurements and data." 


In essence, there 1S some doubt whether eurrentctgreserve 


drilling maintains even the simplest of general military 
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skill, much less the technically oriented skill areas. 
Captain Mailhorne's views are shared by many senior officers 
in the reserve community, and the consequences of neglecting 
the expertise these views express could seriously degrade 


reserve readiness. 
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Fleet Anti-Submarine Warfare Training Center, Pacific 


San Dleqovmeal 1 fornia 
Paul Asa-Dorian, Director of Curriculum Development and 


Standards (AUTOVON 225-4416) 


Dase Dorian has been active in the training area for 
over twenty years, and has often cooperated with the Naval 
Personnel Research and Development Center (NPRDC) on joint 
effort studies. An example is skill retention of weapons 
systems Operators, especially sonar and various other acoustic 
sensor operators. He has developed a modular training system 
eorssuürrace Sonar operators utilizing self-paced instruction. 
This method is extremely effective in initial training of 
Ars E tour sonar tecnnicians (ST), but has proven less than 
adequate for those personnel returning to the fleet for 
second or subsequent tours. Like many other ratings, ST's 
have little chance of maintaining their operational skills 
while on shore duty, unless they are assigned as instructors 
at an ASW training center. The ensuing skill loss from disuse 
is difficult to measure a posteriori and nearly impossible to 
predict beforehand. 

Mr. Asa-Dorian's problem is where to place personnel 
Phone modular system Woon their return from billets out 
of the sonar skill area. He believes some sort of performance 
measurement could prove beneficial in supplying data for the 


amount of retraining required within the modular system. 
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Naval Occupational Development and Analysis Center (NODAC) 


Washington, D.C. 


Mike Callahan, Navy Occupational Development Task Analysis 
Program (NOTAP) Department Head (AUTOVON 288-4621) 


C. T. Marshall, NOTAP Analyst (AUTOVON 288-4621) 


NOTAP was developed out of a requirement for task infor- 
mation data across all Navy enlisted billets. Collection of 
data commenced in 1967, and as of this year, approximately 
ninety percent of the enlisted rates and two officer categories 
have been processed and put into NOTAP data banks. The basic 
ns or Ens accumulation of job data ls to identify what 
K do, not how well they do it. The development for each 
rating proceeds in the following manner: 

1) Subject matter experts (SME) from NODAC familiarize 
themselves with background information such as NEC manuals, 
Occupational Standards, rate manuals and PQS. 

2) The SME team then travels to a specified job site where 
memes tensive use of the particular rating exists to obtain 
direct observations. 

3) A task inventory suvery is then developed and adminis- 
tered to 18-23% of the rating population. 

4) The results are reviewed, then placed on tapes and 
filed in a computer data bank. 

Ti OFE data collected consist ot such things as job 
satisfaction; primary and collateral duties; job characteristics; 
and most importantly, task statements which may be as numerous 
as 600. Access to the NOTAP data is handled by over forty 


subprograms which can analyze the data in various ways. 


65 











e elation, cluster analysis, and matrix reduction are but 

a few of the options available. In essence, NOTAP offers an 

initial start in developing task analysis for a particular 

rate, and additionally some rather useful data analysis tools. 
Unfortunately, task criticality is not addressed as in 

the Air Force CODAP system. Also, user access must proceed 

via the Bureau of Naval Personnel, thus making any local 

real time response difficult, if not infeasible. However, 

the possibility exists of obtaining data tape copies on a 

Specific rating for use at a local level. This would facili- 

tate any in-depth analysis at any Navy facility having computer 


capability. 
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Naval Personnel Research and Development Center (NPRDC) 


San Diego, California 
Adolph Anderson, Code 302 (714/225-2371, AUTOVON 933-2371) 


Mr. Anderson and his colleagues are in the process of 
developing a Performance Proficiency Assessment System for 
the Director of Naval Education and Training (DNET). The 
system will have, as its primary objective, the determination 
of personnel readiness throughout the Navy. To this end the 
system's foundation has the following structure: 

1) Development of Job Performance Measures (JPM): Two 
c pe rates, the Internal Communication Electrician (IC) 
and the Surface Sonar Technician (ST), have been selected in 
conjunction with developing hands-on JPM's. These instru- 
ments will determine if the individual can accomplish the 
mi Cal tasks within his job. 

2) Use of the quality control concept and sampling tech- 
niques: Since it would be expensive to administer JPM's to 
the entire rating population, sampling techniques will be 
employed to determine the degree of readiness within any 
particular rating. The aggregate of all ratings would thus 
quantitively indicate personnel readiness fleetwide. 

3) The prototype phase of the program will be completed 
in FY 1981, and at that time, other ratings will hopefully be 
considered for development. 

It is of interest to note that great emphasis was placec 
on task analysis in initial development, and extensive use 


of the Navy Occupational Task Analysis Program (NOTAP) preceded 
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identification of critical tasks. The Personnel Qualification 
Standard (PQS) material was also used, as was data on Navy 
Occupational Standards. The relevancy of this assessment 
s ICO this study lies in the mutual objective of deter- 
mining proficiency, or said another way, how well an individual 
jmeratns Nis skill. The target populations differ (active 

duty in skill area versus various components out o£ skill 
area), but the concept of a quantitative performance measure 

1S quite amenable to ascertaining the degree of skill loss in 
populations both in and out of previously learned skill areas. 

In discussing skill retention in a more general sense, 
Anderson emphasized the following: 

1) The more rapid a person acquires a skill, the longer 
his retention. 

2) Continuous skills (tracking) are retained longer than 
derete (procedural) skills. 

3) Obsolescence should be considered separate from skill 
deterioration. Skill deterioration usually takes place early 
On in the period of skill disuse while obsolescence is of 
longer term. 

4) The longer the retention interval, the greater the 
skill loss. This especially applies to procedurally oriented 
Skills. 

In summary NPRDC and especially Anderson's group are study- 
ing areas of great mutual interest. Several contacts in other 
agencies were illicited, and continued exchange Cf in orma ior 
concerning job performance measures and assessment systems 


should prove to be quite beneficial. A good synopsis of the 
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NPRDC Performance Proficiency Assessment System was published 


by Pickering and Anderson [1976]. 
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Naval Reserve Readiness Command, Region Nine 


Memphis Tennessee 
Captain D. A. Nystrom, Commanding Officer (AUTOVON 966-5550) 


Because a large percentage of this study's target popu- 
lation is the reserve force, contact was made with a senior 
officer experienced in maintaining reserve readiness. The 
Selected Reserve of 104,000 is organized into sixteen Readi- 
ness Commands. These commands are structured under both 
functional and geographic guidelines. Captain Nystrom's 
command (CTF-9) of approximately 6000 men is basically surface 
ship Oriented and stretches from Kentucky down through Florida. 

ace the initiation of the total force concept in 1973, 
much effort has been made to increase reserve readiness and 
enhance the ability of the Selected Reserve to augment the 
active duty force. een deficiencies still exist 
in the area of training, specifically skills maintenance. The 
48 drill periods and 2 weeks active duty per year are proving 
to be adequate in the area of general military training, but 
Sor e Of insuring a Sufficient amount of skill proficiency. 
Some of the more serious shortcomings are: 

Doas istent funding for long range training programs. 
Meco rapnical location of training centers within any 
Beadiness Command prevent 100% participation in specific skill 

maintenance. 

3) Most equipment is obsolete. For example, communication 
components used in some training centers is of Korean War 


vintage. 
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4) Very little software or simulator systems to train 
weapons systems Operators. In most cases, if personnel from 
the surface Selected Reserve were called up, they would have 
little or no knowledge of modern fleet systems. 

Captain Nystrom discussed a new program, the Versatile 
Reaamming System (VTS), which was initiated in 1972. Oriented 
towards enhancing enlisted aircraft maintenance training, VTS 
utilizes software simulation and self-paced instruction along 
with tailoring each individual's training by comprehensive 
performance tests (i.e., diagnostic, pre-tests, post-tests, 
PQS, etc.). Unfortunately, funding for remote terminal sites 
throughout Readiness Command Nine (pilot study) is unavailable 
at this time. Additionally, there is some disagreement as 
to VTS viability in the surface reserve, where many paygrades 


and rates are intermixed within any particular training center. 
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Naval Technical Training Center (NTTC) 


Memphis, Tennessee 


Pepe coolidge, Program Manager for Other Service Veteran 
(OSVETS) Retraining Program (AUTOVON 966-5955) 


The OSVETS program is designed to indoctrinate newly en- 
listed veterans of other services, in lieu of basic recruit 
training, in the procedures, policies, regulations, and 
traditions of the United States Navy. The typical profile 
of the OSVET applicant is an E-4 with two years prior service, 
a "clean" record, and no deficiencies on the Basic Test Battery. 
The course length is three weeks and deals only with general 
weary training such as firefighting, damage control, ship- 
board safety, and human resources management. The two OSVETS 
training sites, San Diego and Great Lakes, have a combined 
average annual input of 3500 to 4000 people. 

Mae intent of OSVETS, that of recapturing personnel re- 
sources, is quite sound. Although no specific skills are 
retrained, efforts are made to place OSVETS graduates into 
ErEröpriate technical schools, particularly those located 
within the NTTC complex at Memphis. 

However, there are no quantitative methods to aid in 
screening OSVET applicants. In lieu of this, if the period 
of non or broken service exceeds two years, the individual is 
automatically sent back to recruit training. In addition, no 
performance measure is used to determine to what degree the 


OSVET graduate is qualified for technical training or 
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retraining. Here again skill retention data and methods to 


measure performance would greatly enhance the quality of the 


OSVET program. 


73 





Seville Research Corporation 


400 Plaza, Pensacola, Florida 
Dr. Wallace W. Prophet, Director (904/434-5241) 


Dr. Prophet was contacted because of his past work on 
retention of flying skills. Under the auspices of the Studies 
and Analysis Branch of USAF Headquarters, he authored an ex- 
tensive literature review and annotated bibliography mentioned 
in the preceding section [Prophet, 1976]. Although his present 
BESSaren Group 1s not working in this area, he had several 
relevant comments. First, any retraining program must take 
s C Count the problem of individual differences. There 
are many variables comprisinq each person's nonutilization 
period which can interact in various ways to produce numerous 
retraining needs. He felt skill decay may be siqnificantly 
checked through the use of low fidelity rehearsal devices. 

This type of program would especially be amenable to practicing 
procedural skills which are the most rapidly lost. In order 
for this idea to be successful, it was stressed that the 

Navy must provide sufficient incentive to motivate each in- 

Fu uq lr co keep current in his rate. Finally, Dr. Prophet 
agreed that performance assessment would be a useful approach 
to determine how much retraining was necessary provided a 
military agency developed the program to ensure its usefulness 


related to the operational environment. 
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U. S. Air Force Human Resources Laboratory 


Brooks AFB, San Antonio, Texas 


Dr. Raymond E. Christal, Occupation and Manpower Research 
Division (512/536-3648, AUTOVON 240-3648) 


For many years Dr. Christal has been a leader in research 
concerning enlisted skill acquisition, classification, and 
performance. One of his major achievements was creation of 
the Comprehensive Occupational Data Analysis Programs (CODAP) 
which is a computerized system for organizing and analyzing 
Po rce occupational information. Among other things, the 
system is capable of storing component tasks for each tech- 
we rate along with the relative task criticality. Critical 
tasks are identified by examining three factors; a) consequences 
of bad performance, b) task delay tolerance, and c) task 
meaning difficulty. Difficulty is defined as time to learn 
and is thought to be related to deterioration of the task. 

The primary data base is a task inventory conducted every six 
years. At this time Air Force jobs appear to be quite stable 
regarding the types of skills they require. Because of the 
CODAP system's proven usefulness, other services have adopted 
Eilean programs and the Navy system, NOTAP, will be discussed 
subsequently. 

Fa -nascas Dr. Christal is not currently working on any 
Broject directly related to this study, his past experience 
Ener ucrumesplans are worth mentioning. His main interest is 
exploring the usefulness of aptitude tests which he recently 
discussed in a paper presented to the Military Testing Asso- 


ciation [Christal, 1976]. The main hypothesis is that 
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aptitude is a measure of the speed of skill acquisition. A 
eerollary is that aptitude is also, then, a measure of re- 
earning time. Although it is felt that skill deterioration 
may be unrelated to aptitude directly, there are indirect 
connections between skill acqusition and skill decay. In this 
regard, Dr. Christal is beginning work on what he calls 
"perishability of skills." Past work supports the belief 
that those who learn faster and therefore attain greater 
proficiency, lose their skill level less rapidly. A future 
study will attempt to measure learning and decay and relate 
these to the Air Force aptitude testing battery. The question 
to be answered is the following: Is aptitude a good differ- 


ential for learning and decay rates? 
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U. S. Air Force Human Resources Laboratory 


Williams AFB, Phoenix, Arizona 


Dr. Edward E. Eddowes, Senior Technical Assistant 
(602/988-2611, AUTOVON 474-6604) 


Dr. Eddowes has devoted much of his energy toward pro- 
ficiency of flying skills. After the basic psychomotor skills 
involving flying are acquired, the concentration of training 
is on procedural type skills, not unlike those associated 
with Navy technical enlisted rates. It is agreed by most 
authorities that procedural skills deteriorate much more 
rapidly than psychomotor skills. Dr. Eddowes has recently 
completed a proposal for research on maintenance of flying 
skills. His attack focuses of reacquiring lost skill and on 
L amount of training sufficient to do the job. 

On the subject of performance assessment, Dr. Eddowes 
believes that the learning experience depends on viable per- 
formance evaluation to be successful. Because the system 
operator gets feedback from the system, the system output 
approach to assessment is often preferable to the control 
input approach. In other words the former approach measures 
final system behavior while the latter looks at what the 
Operator does to the system. In essence an assessment program 
must first identify and define the skills involved, and then 
develop a means of measurement of those skills. This sequence 
of events will insure the relevancy of the performance measure 


o the task. 
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U. S. Army Research Institute (ARI) 


Ae canaria Virginia 
Dr. Joyce Shields (AUTOVON 284-8695) 


Dr. Shields is working on a skill retention study involving 
Chaparral missile technicians. The Army has found this par- 
ticular MOS is lacking in demonstrated job proficiency in an 
operational environment. In order to answer questions in 
this area,a study has been designed to accomplish the following: 

1) Evaluate skill loss between training and utilization 
en ehe job. 

2) Determine the most effective refresher training. 

3) Provide data on forgetting over time for task performance. 

Ines seucyz design 1s. longitudinal in- nature, and utilizes 
comparisons between control and noncontrol groups over varying 
Perention intervals of zero, one, two, and four months. The 
basic intent is to gain data on the effects of nonpractice of 
a previously learned skill, which closely parallels the present 
effort of this paper. The maximum retention interval of four 
months is somewhat less than what is desired here, but other- 


fase the Similarity is quite significant. 
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U. S. Army Training Support Center 


Ort Eustıs, Virginia 


Capt Harry Porthouse, Individual Training Evaluation 
Group (AUTOVON 927-3128) 


The Training Support Center oversees and administers the 
Army's new Skill Qualification Test (SOT) program. The SOT 
replaces the former norm-referenced MOS test with a criteria- 
referenced performance battery. The program is essentially 
a quality assurance device designed to maximize combat 
effectiveness. It is also used to directly support training 
and the Enlisted Personnel Management System. SOT is based 
on the critical tasks identified in the Soldier's Manual and 
1s administered in as many as three parts; written, hands-on, 
and performance certification. The program is very well 
Organized and documented as demonstrated by the following 
rees: Ford, Campbell, and Harris 11976]; Maier, Young, 
and Hirshfeld [1976]; and Taylor and Vineburg [1975]. The 
Army seems to have the right idea by developing a compre- 
hensive program capable of use for performance evaluation, 
readiness assessment, personnel management, training, and 


advancement. 
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De RESEARCH GAPS 


While investigating past literature and agency sources, 
several neglected areas of research as related to this study, 
have become apparent. In this light, the following points 
are presented: 

Dy in the area of skill research, very little effort 
has been devoted to the subject of skill retention. 

>, Meee teetaining data, excluding flight skili Teten- 
tion studies, considers learning a new skill as opposed to 
F I i ring a previously acquired skill. Few reports exist 
EOS of skills due to the nonutilization of those skills. 
3: Methods of assessing performance have only begun to be 
examined. “There is little data on the selection of performance 
ateria, and on identifying critical tasks. 

4. There has been insufficient consideration of in- 
dividual differences in operator performance (e.g., individual 
aptitude in relation to retention). Research on general learn- 
ing and relearning curves is not meaningful until the factors 
affected by individual differences are investigated. 

e Little work has been conducted concerning predictive 
measures of skill acquisition and retraining. An example is 
the relationship between aptitude and learning. There has 
been no specific study of aptitude and skill deterioration 
Or retraining. 

6. Many factors have been hypothesized to affect skill 
retention during a period of disuse; however, few studies 
Mewemerled to quantify those variables. For example, inter- 


ference from the activities conducted during the nonutilization 
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period is thought to be a major cause of skill decay. With 
Be exception of a brief study on flying skills, this area 
remains relatively untouched. 

I The Navy currently has several job classification 
systems: Personnel Qualification System (PQS); Occupational 
Standards; Navy Enlisted Classification (NEC); Navy Occupa- 
tional Task Analysis Program; the rating structure; etc. 
However, there exists no standardized operational taxonomy 
for fleet-wide use. 

8. Finally, most of the research conducted to date has 
been in a laboratory setting. Few efforts have studied skill 
acquisition, retention, and retraining under actual complex 
Operational environments. Herein may lie the most grave 
deficiency. Many authorities on applied research in these 
areas agree that much of the work done previously may not 


transfer to operational problems. 
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IV. CONCEPTUAL MODELS 


A. MOTIVATION 


The previous discussion of informational sources has 
suggested several areas related to skill deterioration and 
Beeraining that require initial or additional research effort. 
Answering the retraining question is infeasible while know- 
ledge in these areas remains deficient. With this in mind, 
the topic remaining to be addressed is the development and 
Organization of a retraining program in the Navy. In order 
to fully understand the total retraining problem, the approach 
taken must not only seek to close the above individual gaps, 
but also should strive to examine the relationships among 
these areas. Specifically, the interactions of the various 
factors which contribute to skill deterioration and influence 
retraining in the operational environment need to be investi- 
gated. For example, several interactive variables are in- 
dividual differences, task type, length of the retention 
period, and retention interference factors. 

Thus, one viable approach for discussing a retraining 
program is the development of conceptual models. The model, 
in this case, represents an advancement from the present 
State-of-the-art of enumerating past deficiencies to an 
Sagan zacon Gt a plan describing future solutions. Concep- 
tual models are efficient methods of presenting a problem 
comprised of several integrated parts. They include a 


sequential plan describing various stages of development, 
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depict functional relationships between those stages, and 
provide an organized decision pathway for implementation. 
nally, information which is requrred prior tower duning 
model development and data supplied after it becomes validated 
andscoperatiıonal, should be utilized for closing the research 


gaps and further defining their interrelationships. 


Dr THE MATRIX APPROACH 
T. Background and Definition 


Anemia Ei, OF grid concept, is used trequently 1n 
mathematics to correlate two or more variables which have 
specified value sets. In the social sciences, the matrix 
approach has been used to equate one set of behavior charac- 
teristics with another. Basically, the grid concept can be 
used wherever knowledge of classification, correlation, 
comparison, or interactions needs to be investigated. The 
matrix approach herein will be used as a decision-maker's 
tool for determining, or in some respects, predicting skill 
deterioration and retention across the broad spectrum of 
Navy ratings. 

For simplicity, the matrix will be two-dimensional, 
eaadewillsequate job clusters with job characteristics/skills. 
Figure 2 serves as an example of clustering jobs and job 
emac teerist tics. Along the vertical axis are the seventeen 
occupational categories as outlined in the Navy Recruiting 
Manual, which are functionally classified. The horizontal 
axis dimensions job characteristics which require various 


abilities, but are not single skills per se. The intersection 
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of one Job characteristic with one job cluster indicates the 
interactive effects. The cumulative effect of all Job 
characteristics on any particular job cluster may be deter- 
mined by observing the total row composition of the respective 
job cluster. Conversely, the column composition will indicate 
the effect of one particular job characteristic on all job 
clusters. The question of cell scaling now arises. Each 
individual cell may be thought of in one of two ways: 
1) The degree of importance, on a percentage scale, 

Ze Per reulorZjobscharaeterjstieunnasson a Job cluster. 

The row total for any job cluster would then sum 

to 100 percent. 

2) The severity of deterioration, on an ascending 

scale of one to ten, a particular job character- 

istic has on a job cluster. The row average could 

then represent a composite factor of skill deteri- 

aration for the respective job cluster. 

Even though only two dimensions are used in the 
Matrix, a third dimension depecting the level of individual 
experience could be helpful. However, instead of making the 
matrix three dimensional (and more complex), enlisted pay- 
grades will be categorized into three groups: El-E4; ES-E6; 
and E7-E9. A separate skill deterioration matrix as in Figure 
2 will then be done for each group. 

amally, the length of the retention interval must 
be taken into account. Four years of non-use will produce 


more deterioration than will six months. Some multiplier 
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or scalar, perhaps derived in exponential fashion, can be 
used to differentiate cell values on a specified time basis 
(i.e., quarterly or yearly). 

2 Model Description 

The skill retention matrix, by itself, is of little 
value. It must be made an integral part of the operational 
personnel cycle. Figure 3 shows the functional relationship 
Sueno matrix within the cycle. The following is a brief 
description of how the model works. 

a. Selection: The present recruiting system, using 
occupational specialties as depicted in Figure 3, represents 
Amt sStantial effort to match aptitude with training. How- 
ever, the selection process itself is beyond the scope of 
this study, and the only reason for mentioning the occupational 
specialties was their use as a potential matrix dimension. 

aal Training: Tas inelides all training prior 
to operational assignment: recruit, “A” school, and fleet 
replacement training. The only time personnel will pass 
through this phase more than once is to acquire a new skill, 
can rare instances, for complete retraining. In the former 
case, the cost would be prohibitive, unless the individual 
showed unusually high aptitude for the respective training. 

e Duty Tours: The three duty tour categories en- 
compass almost the entire target population. Only those per- 
sonnel on short periods of temporary duty would not be con- 
Sudered. An associated tour would be in the skill area of 
initial training, and every effort should be made to place 


AS Ecol: personnel within their operational rating. A 


86 


Be 





—1  PERSONNEL FLOW = = -o- m INFORMATION FLOW 


SEBECTLON ea = m 
Bee - 


ss Ñ 


INITIAL 


TRAINING ee 










DUTY TOURS 


RESERVE DIS ASSOC. 
ASSOC. 


RETRAINING 


Figure 3. Matrix Model Description 


3 7 





disassociated tour would be out of the primary skill area, as 
often happens to individuals rotating from sea to shore tours. 
m reserves category is actually an extended disassociated 
tour, but is considered separately because of the inherently 
longer periods of retention. 

d. Skill Retention Matrix: The entire model is intended 
for Bureau of Naval Personnel (BUPERS) use, where large amounts 
of manpower are constantly shifted throughout the Navy structure. 
It is at the BUPERS level where the skill retention matrix will 
be most effective, since personnel are detailed from one tour 
to another here. The matrix will reflect previously learned 
Skill areas, length of non-use intervals (if any), and exper- 
tence level, to ascertain the amount of retraining necessary 
to facilitate re-entry. Those personnel having just completed 
an associated tour would be directly detailed without use of 
the matrix, unless a record of inadequate performance is 
noted on periodic evaluations. The matrix will be especially 
useful in dealing with large groups, such as reservists coming 
back into active service to augment the regular force. The 
intent of the matrix is to provide the Navy with a method by 
which skill deterioration can be predicted without the prior 
necessity of extensive longitudinal studies. Admittedly, the 
initial iteration of the matrix scheme will represent a "gross 
cut" at the problem, but later evolutions should prove more 
and more accurate. | 

e. Retraining: After the retention matrix has indicated 
the gross level of skill loss incurred, BUPERS can detail the 


individual to the appropriate retraining level. Presently, 


88 





there exists on-the-job training, fleet replacement training, 
technical training (A, B, and C schools), and recruit train- 
ing. inherently, this represents a hierarchy of training 
Taverns, and could be modified to incorporate retraining as 
well. However, besides the costs ensuing from program modi- 
K (ions to facilitate retraining, the logistical costs must 
also be considered. To use existing sites for retraining 
would require vastly increased amounts of temporary personnel 
movement, billeting requirements, and temporary additional 
duty funding. One alternative to this is establishing re- 
training sites in close proximity to areas of large personnel 
concentration, such as San Diego and Norfolk. Procurement 
costs for these remote sites would be expensive, and before 
any decision can be made on the method of retraining, a cost 
analysis must certainly be undertaken. 

f. Re-entry: This is the juncture where the individual 
re-enters his previously learned skill area, presumably re- 
mened and again proficient in his specialty. It is also 
during the first few months after re-entry where data on job 
performance should be collected. 

g. Data Collection: To prevent confounding the effec- 
tiveness of retraining (and the retention matrix) with the 
benefit of being on the job any more than a few months, data 
collection on job performance should be done soon after re- 
entry. At present, the enlisted performance evaluation is 
the only method, and it is a subjective measure not free from 
bias. Some measurement methodology need be developed to 


offer an objective quantifiable indication of job performance, 
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such as hands-on performance tasks, or criteria based paper 
and pencil testing. In any case, the information gained from 
data collection will have the Eolo implicacions on the 
entire model: proper aptitude criteria for selection of 
personnel; amounts and types of initial training required; 
updating the matrix dimensions and cell scalars; and determining 
Enerefrzectiveness of various retraining levels. 

94 Matrix Implementation 

As was mentioned before, the initial matrix will 
serve only as an estimate of skill loss, with improvements 
certainly to be made as feedback dictates. The following 
discussion will supplement the implementation diagram in 
Egurre 4, 

a. Documentation Survey: The Navy has several 
personnel classification systems. The Navy Enlisted Classi- 
fications (NEC), Occupational Specialties, and the Rate 
Manuals are primarily requirements and qualifications docu- 
ments, while the Personnel Qualification Standard (PQS) is a 
procedural instruction system for actual operational use. 

The Navy Occupational Task Analysis Program (NOTAP) deals 
cose content and maintains an occupational data bank on 
all Navy ratings. The intent in reviewing these systems is 
to develop sufficient background in support of proposing 
alternatives for the matrix dimensions. 

D: Matrix Dimension Alternatives: As was mentioned 
Mene definition of the matrix approach, Figure 2 serves as 
an example set of dimensions. There are other conceivable 


alternatives: NOTAP job cluster analysis versus Fleisnman's 
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[1967] psychomotor and physical performance Faetors: or 
clustering of Navy ratings contrasted with categories of 
perceptual, mediational, and response processes. Again, the 
possibilities are numerous, but the dimensions selected should 
reflect operational experience, and not merely empirical 
laboratory results. Also, the dimensions should be of workable 
size, with a tractable number of categories per dimension. A 
Fereifie figure would be difficult to assign currently, but 
perhaps bounding the range between ten and fifty would be 
sufficient. 

Cx Navy Occupational Development and Analysis 
Center (NODAC) Panels: this is the key step in matrix imple- 
mentation. Subject Matter Experts (SME) will either confirm 
or modify the matrix dimensions, and assign scalar quantities 
to the matrix cells. Since the SME's will be experienced 
senior enlisted members, their inputs will greatly affect the 
implementation of the matrix. Furthermore, their experience 
in task analysis should be of use in validating the construc- 
tion of the matrix, especially if the dimensions previously 
Selected are job clusters and job characteristics as depicted 
mimeeicurce 2. 

d: BUPERS: the first iteration of the matrix will 
then be sent to BUPERS for approval and dissemination to the 
detailing section and other departments. Of course, primary 
use of the matrix will be to include retraining considerations 
in the detailing process. BUPERS will also coordinate with 
the Chief of the Naval Reserve when reserve units are tasked 


with active force augmentation. As consequent iterations of 
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the matrix cycle evolve, BUPERS will act as a central contact 
point for the Chief of Naval Education and Meaining, the 
Chief of Navy Recruiting, and other agencies to insure the 
matrix is in alignment with the personnel cycle from the re- 
cruiting through retraining stages. BUPERS can also use 
matrix generated data as inputs for the Navy and the DOD 
training budget. 

e. Collection of Data and Matrix Update: the 
success of the matrix depends upon, among other things, feed- 
BER information to insure flexibility and continuity. The 
data collected on personnel after re-entry into their skill 
areas will serve to adjust the matrix cell quantities, and 
also verify the previously selected and validated matrix 
dimensions. As mentioned before in the model description, 
either enlisted performance evaluations or some form of testing 
procedure will be the source of such data collection. 

4. Summary 

MS > Or E ante to again mencion Cna the intral 
Maries an estimate or "gross cut" prediction Of skill loss. 
However, it represents a potential approach to the skill deg- 
Gadation problem, which has to date been largely neglected. 
In essence, the matrix will supply numerical values which 
indicate approximations of occupational skill loss. These 
matrix values will be sensitive to both experience level and 
length of nonutilization period, and subsequent iterations 
will serve to refine the net approximation. Thus, the matrix 
will provide the BUPERS level decision-maker with a skill 
degradation estimate as the initial step in determining 


retraining requirements. 
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The matrix is the foundation of the model depicted 
in Figure 3. The other functional blocks presently exist in 
varying degrees of effectiveness. No attempt has been made 
to critique them since that is not an objective of this study. 
Because of this, only the matrix portion of the model is 
carried through development of the implementation plan. 

Also not fully developed was the testing methodology 
for data collection after re-entry. It is perhaps hazardous 
to rely solely on enlisted evaluations to gather such infor- 
K ron and thus,effort to construct an adequate performance 


measurement is indicated. 


= PERFORMANCE ASSESSMENT APPROACH 
L. Background 

After perusing the preceding sections of this paper, 
mars hoped that Be has gained an appreciation for the 
usefulness of job performance measures (JPMs). Modern criter- 
ion referenced performance tests can directly evaluate in- 
K II] proficiency. Since job performance is the output of 
training, JPMs should be capable of assessing the training 
process as well. In other words, the capability of JPMs to 
evaluate proficiency means they can also identify areas of 
weakness. These areas can then be translated into inputs for 
s i haus, the relevancy of job performance measures 
to both proficiency assessment and training requirements, 
Ano tdesehe basis for this conceptual approach. 

Since the ultimate question to be answered concerns 


the amount of retraining necessary after a period of skill 
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deterioration, it would prove useful to examine the advantages 
that job performance measures could offer. A JPM system 
measures current proficiency, and therefore can identify 
current training needs. By administering an up-to-date JPM 

to an individual prior to his return to a previously learned 
skill area, specific training units can be prescribed for any 
deficiencies. This capability reduces the need to be con- 
cerned about the nonutilization period and the retention 
variables involved. The most relevant question for retraining 
is one of how well the individual can perform his tasks at 

m resent time. Additionally, obsolescence is another 
variable of the retraining problem which would be minimized 

by a JPM approach. Provided the JPM is updated as technology 
advances, it would always evaluate performance at the current 
State-of-the-art regardless of the amount of obsolescence 

that had occurred. A JPM program would also provide stand- 
ardization for evaluation of instructional objectives and 
Operational performance. It would force the two communities 
of training and readiness assessment to adopt the same criteria 
for measurement. It is intended that this model give the 
decision-maker a tool that will aid in answering the retraining 
Begutnhements question. A major portion of this tool is the 
JPM itself which provides a direct, objective measure of 
proficiency. This type of conceptual approach combines 
readiness assessment and retraining assessment into one 
system. Additionally, it provides feedback as well as data 
For training programs and future research respectively. A 


widespread performance measurement system is especially useful 
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as a manpower management tool in the areas of selection, 
placement, and advancement. In essence, with this model, the 
decision-maker can receive the information necessary to insure 
that the proper numbers of qualified personnel are assigned 

to the fleet. 

In the sections to follow, the conceptual model will 
be treated in detail and a recommended implementation sequence 
will be presented. The model is designed primarily to describe 
a career training/duty cycle which may be interspersed with 
periods of nonutilization. The JPM approach can be applied 
to active duty or reserve personnel. Additional discussion 
concerning the implications for reservists will be contained 
at the end of this section. 

2. Model Description 

The basic JPM model is composed of two sites: a 
training site and a duty site. An assumption of the model is 
that the same JPM be given at both sites. This approach 1s 
Mot viable d£ differing criteria, hence, differing proficiency 
measures are used. Also, the JPM utilized must be technologi- 
cally current. The training site cannot produce competent 
personnel for the duty site if obsolescence is allowed to 
come between them. 

The model is depicted in Figure 5. A description 
of the model is organized into the following four phases: 
initial training, the JPM, retraining, and duty site. 

a. Initial Training 

The initial training block is where the indiv- 


ual obtains basic and advanced training for his pari icujaox 
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rate. The model operates under the assumption that each 
person is initially trained only once for each rate or skill 
area. If for any reason the individual changed rates or is 
taught a completely new set of skills, he must enter the model 
through the initial training block again. The training program 
should use the same criteria for construction of instructional 
objectives as are used to develop the JPM program. The output 
Si Nes initial training section goes to the JPM section. 

b. Job Performance Measures (JPM) 

The JPM section is the focal point of the entire 
model. Development of adequate performance assessment tools 
is receiving much attention throughout the military, as was 
mentioned in previous portions of this paper. Although it is 
beyond the scope of this study to construct a prototype JPM, 
the following ideas are contributed in order to aid future 
efforts and to emphasize crucial assumptions of this model: 

1) The JPM battery should be developed from 
Criteria based on task analysis at the operational level. In 
addition, supervisory personnel can help identify critical 
tasks. The final set of proficiency criteria must make sense 
to the worker himself. 

DWERE thefrisk osnbeingseredundantrertzisgre- 
emphasized that the criteria developed for performance measure- 
ment must also be used as a basis for constructing instructional 
es eckives for the training portion of the model. It as essen- 
tial for the individual to be trained in the same skills that 


he will use operationally in order for this approach to work 
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efficiently. A separate JPM battery should be developed to 
evaluate the particular skills of each rate. 

3) The JPM can utilize both written and hands- 
on methods of assessment techniques, depending on the types 
of skills involved in the particular job. 

4) The types of skills utilized also deter- 
mine whether a process or product method of assessment is 
preferable. The former approach monitors the operator's 
actions while the latter method evaluates the results of his 
actions. In most military settings the task is complex (i.e., 
sonar operator, radar operator, etc.) which necessitates 
evaluating the product. Reference 34 contains a thorough 
G@eeseussion of this topic (p. 8-11). 

5) To be effective, each JPM should be as 
objective as possible. Objectivity is the ability of the 
test to perform consistently under various evaluators or 
under one examiner for various subjects. 

6) Finally, every JPM must be kept current 
with operational requirements. The JPM will need to be up- 
meeeamwhen significant technological or procedural advance- 
ments occur. This implies that the training sites must keep 
abreast of operational changes also. Ideally, the training 
sites should be the first to receive new innovations in order 
to ensure the fleet is provided with highly qualified personnel. 

As depicted in Figure 5, the JPM block provides 
an interface between the initial training, retraining, and 
duty site sections. Inputs coming from both eLo Aa An 


de raining are utilized for a comprehensive, practical 
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assessment of performance upon completion of an instructional 
period. Also, a diagnostic job performance evaluation is 
conducted on personnel coming from disassociated duty sites. 
Personnel with substandard performance from the above three 
sources are sent to the retraining section along with a 
breakdown of specific deficiencies. Upon satisfactory com- 
pletion of the particular JPM battery(s) for the applicable 
rate, the individual is detailed to his next duty assignment. 
Ada tlional specifics concerning JPMs will be presented in 
the implementation section. 
Ch Retraining 

Tmewretraıinıng block is cheszenard portion of 
the training site. It receives its inputs entirely from the 
JPM block. The retraining program consists of many separate 
training modules or units. These units are comprised of 
related skills required for specific rates. Since one of the 
assets of a JPM system is its capability to identify deficient 
performance areas, each trainee can be assigned to complete 
only those units where retraining is deemed necessary. Simu- 
lation and self-paced instruction techniques would seem to 
lend themselves well to this type of training situation. 

Upon completion of the required number of re- 
a ining nits, training data will be collected and a JPM 
battery will be administered once again. The types of data 
that may prove useful include information on the type and 
e eri nmonutilization period (if applicable), initial 


training scores, JPM scores, types of retraining units given, 
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etc. A JPM reexamination would insure attainment of minimum 
proficiency standards as well as provide feedback for the 
retraining system. 
ae Duty Site 

The duty site section of the model represents 
all permanent duty assignments an individual might receive 
during his naval career. The duty site is divided into asso- 
ciated and disassociated tour types. These terms differentiate 
respectively between assignments related to the rate for which 
the individual was initially trained and assignments where the 
skills of his rate are not used. Thus, a disassociated tour 
can be equated with a period of nonutilization as defined 
and discussed in earlier portions of this paper. 

Inputs to the duty site can Come from success- 
SW ISrion or tralning evolutions via the JPM block or 
G r revious associated tours. During associated tours, 
personnel will periodically be given a JPM battery with OJT 
to correct any inadequate job proficiency. The individuals' 
training records will also be kept current. An individual 
completing a disassociated tour will need to be given a JPM 
evaluation to determine the type of retraining necessary, if 
any. In order to administer this approach, BUPERS must be 
able to distinguish between associated and disassociated tours 
© duty- 

Er @ommunication Channels 
The magnitude and complexity of the retraining 

problem dictates that communication channels be briefly dis- 


cussed. Figure 6 is one proposal for establishing lines of 
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communication. It is necessary there be an interface between 
the training site and the duty site for exchange of informa- 
Gren and feedback. Thus, a Training Coordinator was inserted 
between the two major sites. This coordinator will receive, 
assimilate, store, and disseminate data from all other sections 
ene smodet. Thus, it is hoped that each site will receive 
timely information regarding changing performance require- 
ments. An educational system must receive constant feedback 
from the users of its product in order to fine-tune its in- 
structional methods. Training and performance data is col- 
Meered at both sites and sent to the Training Coordinator 
where it is compiled for informational and research purposes. 
For example, JPM data could have implications for establishing 
future selection criteria for technical enlisted rates. Man- 
power management information is also sent to BUPERS where 
decisions are made regarding personnel placement at the duty 
Site. 
4. Implementation 

This section will outline a possible implementation 
plan for the JPM model. The plan is depicted in Figure / and 
is discussed below. 

a. Task Analysis 

To begin, all tasks for each rate must be 

identified. This can be accomplished by using the Navy Occu- 
pational Task Analysis Program (NOTAP), the Personnel Qualifi- 
cation Standards (PQS) Program, and specific job manuals. 
The list of tasks must then be ranked according to criticality. 


Several factors to be considered when identifying critical 


Oe 











TRAINING SITE 
BUPERS 


INITIAL 
TRAINING 


DIAS ETE 







DISASSOC] ASSOC. 
LOUR TOUR 






DATA 
GCOLEEGLEON 


DATA 
COLLECTION 


| 

| 

RETRAINING | 
UNITS i 
| 

| 

| 

l 

| 

| 

| 

| 


Bigure 6. JPM Communication Channels 


103 





JPM DEVELOPMENT 


«Select Assessment Type 


-Construc*<* JPM Units 


«Validate Units 


JPM ADMINISTRATION 
-Standardize Procedures 
‘Standardize Scoring 


Establish Qualification 





Standards 


fea slis ESTABLISHMENT 


— 


-Duty Sites 


«Training Sites 





TASK ANALYSIS 


"Identity Tasks 


Assess Critical cy 


CRITERIA IDENTIFICATION 
«State Skills 
“Speci ry Environment 


«Identify Performance 
Standards 












TRAINING OBJECTIVE 
DEVELOPMENT 


«Identify Behaviors 






INITIAL TRAINING 
CURRICULUM DEVELOPMENT 


-Realign Curriculum 


«Specify Equipment 





TRAINING SITE 
ESTABLISHMENT 


.Instructors 


«Equipment 


DATAS COLLECTION PROCEDURES ESTABDISHMENT 


“Training 


Coordinator «Training Sites 


-Duty Sites *BUPERS 





Deure /. JEHZImplementarremer Lan 


104 


RETRAINING UNIT 
DEVELOPMENT 





“Construct Units 


‘Specify Equipment 











tasks are the following: 


1) Consequences of poor performance 

2) Visibility of poor performance 

3) Task delay tolerance 

M Task difficulty 

5) Frequency of performance 
A panel of subject matter experts or experienced supervisors 
can aid in selecting and ranking critical tasks for performance 
evaluation, 

Dr Criterion Identification 

Criteria of -standards af performance, are 
derived from the critical tasks identified in the preceding 
stage. These measures of effectiveness must relate directly 
to the actual job tasks they are designed to evaluate. De- 
veloping performance criteria early insures common use by 
Been training and job proficiency experts. 

Three areas must be included in each set of 
complete criteria. First, the skills or knowledge required 
to successfully complete the task must be specified. This 
is basically a statement of what the individual must do. 
Second, the environmental conditions should be identified. 
For example, servicing an aircraft oxygen system is a critical 
task for the AME rate, and the criterion must delineate whether 
the task is to be performed indoors or out, day or night, on 
shore or at sea, etc. Third, the standards against which 
performance will be measured need to be defined. This portion 
of the criterion specifies quality standards and time con- 


Seraints. 
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Se. JPM Development 

Once the criteria have been established,a set 
of JPMs can be constructed. The best method of evaluation 
also needs to be decided. For example, physical maintenance 
tasks may lend themselves well to simulation or hands-on- 
measurement while cognitive type tasks would be better scored 
by written methods. Another developmental suggestion is to 
design each component JPM to evaluate only one critical task. 
Proficiency demonstrated for each task could then be given 
simply a PASS/FAIL score. Additionally, it is recommended 
that several JPMs for a given task be designed both to add 
m ron lıcy in constructing total test batteries and to provide 
a broader base for evaluation and selection of the best JPM 
Pus in constructing the JPM modules, constraints such as 
manpower and equipment availability, costs, and time, need 
to be considered. 

Following creation of a JPM battery for each 
technical rate, the tests must be validated. Checking exam 
reliability encompasses assurance of consistent JPM infor- 
mation output. Validity is the degree of job assessment 
accuracy. Finally, proficiency standards must be established 
relative to the resulting JPM score. In other words, the 
performance scores must be ranked on a’ qualitative scale. 

ar JPM Administration 

In this stage of implementation, procedures 
for administering and grading JPMs must be established. Since 
many sites could conceivably administer JPM batteries, a set 


of standardized procedures for using JPMs must be employed 
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at all locations. This will ensure system continuity and 
the production of useful training data. Testing personnel 
must also be provided with guidelines for scoring and reporting 
the results of each JPM battery. To facilitate grading a 
cut-off performance score for each unit should be established. 
Performance at or above the required level would receive a 
K score. Admittedly, fixing qualification standards is 
rather arbitrary;however, factors affecting the decision are 
task criticality and required manpower levels. A numerical 
grade for the total battery can be derived by simply forming 
ime ratio of passed units to failed units. 
e. JPM Site Establishment 

As the JPM batteries are completed, implemen- 
tation sites must be identified and manned. Optimistically, 
it is hoped that JPM sites can be established at or near all 
associated duty sites for routine, periodic performance 
assessment. Additionally, sites are required at the related 
training establishments for retraining placement. [In this 
case variables such as location of existing training facilities 
and the availability of needed hands-on equipment must be 
considered. 

ES Training Objective Development 

Coincident with the development of the JPMs, 
educational specialists can take the performance criteria 
and write instructional objectives. These objectives are 
usually behavioral in nature which means they identify desired 


responses by the trainee under given situations. Essentially, 
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they define the behavior desired upon completion of Era nein g - 
Behavioral objectives form the basis for curricula development. 
El Initial Training Curriculum Development 
1. a majomkry o cases, curricula for technical 
rating programs of instruction are already developed and in 
use. However, the training programs must be examined and 
realigned to reflect the newly constructed criteria-based 
instructional objectives. 
h. Retraining Unit Development 
Constructing retraining modules is a large, 
but necessary task. Since the JPM batteries will be designed 
to evaluate proficiency levels of specific critical tasks, 
each retraining unit must also cater to a critical task or 
group of associated tasks. Trainees will be assigned to 
eomplere only the specific retraining Units identified by 
deficient performance. Unlike initial training programs 
where basic theory and general knowledge must be taught, the 
retraining program need only concentrate on refreshing motor 
skills and rehearsing procedural skills. The retraining units 
can also serve to modernize skill levels that have become 
ae raded due to technical obsolescence. 
ae Training and Retraining Site Establishment 
Training sites for all rates are well estab- 
lished. The placement of retraining sites will depend upon 
the location of related JPM and duty sites. A major constraint 
in location planning is the type of equipment required for 
training and evaluation. Ideally, the retraining unit should 


be flexible and compact enough to be utilized at the training 
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Zee sand the duty site if necessary. Like JPMs, retraining 
modules should be capable of moving with personnel and equip- 
ment in the fleet. 

JE Data Collection Procedures 

In order to process all of the information 
made available by the various components of this JPM model, 
several points of data collection, as well as a central data 
bank, should be established. As shown in Figure 5, data 
collection occurs at both the training site and the dub bua 
mae Training Coordinator depicted in Figure 6 could serve as 
eee Central data depository as explained earlier. Also, 
individual training jackets need to be maintained and carried 
between sites. 
5% Summary 

Since one of the largest potential consumers of a 
retraining system is the Naval Reserve, it is beneficial to 
discuss this model as it applies to their use. A major reserve 
problem is the lack of modern operational equipment. This 
means, unless equipment funding is substantially increased, 
many of the proficiency criteria developed for the active 
duty forces will not adapt to reserve training programs. Thus, 
reserve training and JPMs must be aligned with applicable 
criteria. Because the reserve system is totally dedicated 
Bow eraimmag, the training and cuty sites, in terms of the 
JPM model, are merged at the same location. Therefore, when 
developed, the JPM batteries and retraining units can easily 
be administered at the Naval Reserve Training Units (NARTU). 
Obviously, data collection and individual training jackets 


Ganeaise be managed at individual NARTU locations. 
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Also, there are several agencies contacted 
during this study effort which contain expertise in specific 
areas relative to the implementation or further definition 
of this conceptual model. The following is a list of possible 
future study areas with the agencies which may be of assistance 


for each: 


1) Task analysis - Chief of Naval Education and Training 
(CNET) 


Navy Personnel Research and Development 
Center (NPRDC ) 


Naval Occupational Development and 
Analysis Center (NODAC) 


2) Performance criteria - CNET 
NERDC 
3) Job performance measures - Director of Naval Education 


and Training (DNET) 
CNET 
NERDS 


ee yaaa ees Wem Oe te 
Center 


4) Retraining unit development - CNET 
Reserve Readiness Command, 
Versatile Training System 
(VTS) 

In conclusion, a Navywide JPM system would be difficult 
and expensive to implement; however, the long-term advantages 
would seem to far outweigh the initial investment. From this 
model the decision-maker receives information concerning both 
fleet proficiency and retraining requirements. Readiness data 


is obtained from the quantitative JPM scores at the duty sites 


and can be used to compare or contrast a number of variables 
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throughout the Navy. For example, similar commands can be 
evaluated in terms of total proficiency and personnel of the 
same rate can be ranked for advancement purposes. In addition 
to a numerical score, the JPM indicates exactly which skills 
of a particular rate need retraining. Also, since the JPM 

is administered individually, the decision-maker is assured 
that the model will not have to be extrapolated to meet popu- 
lation differences. Essentially, this approach provides an 
empirically based tool for manpower management. Once estab- 
lished, the JPM system will be administered by local training 
and duty sites. The decision-maker can control personnel 
selection and placement by adjusting the qualification stand- 
ards. For example, if feedback is received from a duty site 
which indicates that the proficiency of incoming personnel is 
not sufficient to maintain operational effectiveness, then 
tightening the qualification standards will increase that 
proficiency. Since the decision-maker can examine JPM data 
from both training and duty sites, he 1s in a competent 
position to make judgements concerning the alternatives 
between increased retraining time and decreased manpower in 


the fleet. 
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V. RECOMMENDATIONS AND CONCLUSIONS 


ls; It has been more than sixteen years since Naylor 
and Briggs [1961] published the first concerted review on 
danl retention. Their suggestions for further research, 
especially in the area of measurement methodology, have not 
been seriously studied. It appears the immensity of the prob- 
lem and the apparent requirement for lengthy longitudinal 
research has prevented potential solutions from becoming more 
than just proposals. To overcome this situation it is neces- 
sary to develop a method which produces a viable short-term 
mover aualitatıvely, with a longitudinally constructed 
quantitative follow-up. Such a method may be realized by 
incorporating the matrix and performance assessment models 
discussed in the previous section. The approaches can be 


contrasted in the following manner: 


MATRIX JPM 
a. Short-term qualitative a. Longitudinal quantitative 
solution solution 


b. General estimate of skill b. Specific measurement of 


loss skill loss 

eA oli cable to large c. Considers individual 
populations differences 

d. Is a predictive device d. Is an assessment device 


SUE lized ac BUPERS PeUtilezec vat tralnrne/aucy 
(high level) site (low level) 


(D 


wich these complementing features in mind, con- 
current implementation of the two approaches would produce 


results both in the short and long term. The matrix, with 


ll. 





its somewhat qualitative foundation, would soon provide the 
Navy with a BUPERS level instrument for determining retrain- 
ing requirements for large groups. The JPM, with its ability 
to gather data quantitatively, would supply feedback infor- 
mation to improve the matrix, and also establish specific 
individual retraining needs. As an end product, the Navy 
would have a skill retention program which operates on two 
levels (BUPERS and training site) with both predictive and 
assessment capabilities. The system would be closed looped 
with a built-in pathway for modification since JPM collected 
data will continually supply feedback information to improve 
the matrix. In essence, the incorporation of matrix and JPM 
approaches will provide positive action now, while allowing 
sufficient time for data base development. 

2E The use of training devices has been shown to en- 
hance learning in both hardware and software intensive en- 
vironments. Unfortunately, such devices are prohibitively 
Seeve ana are becoming difficult to procure in sufficient 
Bilemeeety and quality. For example, the SO00=3 sonar system 
trainer used in ASW training costs in excess of $2 million, 
while the P-3C weapons system trainer costs over $13 million. 
Also, these and most other present generation systems are 
used for active duty personnel only. The reserve components, 
as was mentioned before, are forced to train on equipment 
developed as long ago as the Korean War period. 

To facilitate training equipment procurement for 


both active and reserve forces, lower cost low-fidellty 
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training devices should be utilized whenever possible. 
Because of diffuse geographic locations of personnel, es- 
pecially reservists, these devices should be mobile and 

fully self-sufficient. In conjunction with software/hardware 
increases for the reserves, a restructuring of the drill 
Pas ud active duty period mix is necessary. This’ revised 
pean should optimize the utilization of training devices 
Pimeergiouc the year in preparation for operation of the actual 
system during the active duty period, and take precedence 
even general military training areas. The final product 
would be increased retention of fleet skills. 

Br Additional research in the area of complex procedural 
skill deterioration is necessary. Future studies must either 
be conducted under actual operational conditions or in simu- 
lated operational environments where the researcher can ensure 
the transferability of the data and the results. The time 
has long past when basic research should give way to applied 
efforts to answer specific military skill retention questions. 

4. Ano progr amn developed. towaddmess vahe cerrara probe 
mememust be dynamic. Many shortcomings Of the reserve system 
and training systems in general stem from their lack of ability 
to keep abreast of changing fleet requirements. Approaches 
such as those conceived in the preceding section will also be 
less effective unless they are allowed to keep pace with pro- 
gressing technology. 

3% The skill retention area has attracted the attention 
of several individuals and agencies, in both civilian ana 


military sectors, resulting in a tremendous increase Im 
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investigative efforts. Some method of collafiang thiesimfor- 
mation on a continuous basis is required. Literature reviews 
at random intervals are not the solution. The Air Force, witi 
its system of Human Resource Laboratories, adequately covers 
studies within their service, but falls short of Covering 
external sources. The Navy has several levels of research 
facilities, from the Office of Naval Research (ONR) to the 
Naval Personnel Research and Development Center (NPRDC); but 
apparently has no effective means of centralizing information. 
Contact with the Army was minimal, but it would not be pre- 
tentious to assume that they face similar circumstances. 
It is thus suggested that each service set up a 

liaison office to cross feed skill retention information on 
a continuous basis. For the Navy, such an office could be 
established within the office of Director of Naval Education 
and Training. Skill retention data could then be disseminated 
s the chain of command. In addition, this liaison officer 
could advise DNET on work assignments within military agencies 
to prevent unnecessary and more costly funding for civilian 
Sontracting. 

Ge Performance assessment has been shown to have wide 
applicability. The Job Performance Measure (JPM) discussed 
in the previous conceptual models section can be used to 
take advantage of the commonalities of personnel training, 
readiness, and advancement. These three areas all use dif- 
ferent forms of measurement based on individual and group 
performance, but seek to achieve the same goal. Readiness 


is ascertained by Operational Readiness Exercises (ORE). 


115 





Training effectiveness currently has no cohesive assessment 
procedure, while advancement relies on enlisted evaluations 
and rating exams. The JPM could be used to measure proficiency 
in all these areas, thereby reducing costs and providing con- 
tinuity within the total personnel cycle. 

7: Regardless of the type of program conceived to 
Ever retraining questions, the criteria established for 
performance assessment must match the criteria used for 
training programs. A performance measure is not useful as 
a diagnostic tool for a retraining program if the skills 
evaluated are not part of the program's goals. It is recom- 
mended that the desired job performance criteria be identified 
before developing performance measures and instructional 


objectives. 
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